Skip to main content

Custom Agents

Custom agents give you full control over the voice pipeline. Write Python code, use any library, define custom tools, and deploy as isolated Docker containers — all from the dashboard.
Preview Feature — Custom agents are currently in preview. The core functionality (code editor, AI assistant, Docker deployment, preview) is stable, but some features may evolve as we gather feedback.
Custom agents require a self-hosted deployment. They are not available on Railway or Render.

How It Works

  1. Create a custom agent from the dashboard
  2. Write code in the built-in editor or let the AI assistant generate it
  3. Build the Docker image from your code
  4. Deploy as a persistent worker or Preview for testing
Dashboard Code Editor


Code stored in MinIO → Docker image built → Container spawned


                                            Connects to LiveKit
                                            Handles voice calls

Creating a Custom Agent

  1. Go to Dashboard → Agents → Create Agent
  2. Select Custom mode
  3. The agent is scaffolded with default files:
agent.py              — Main entry point (required)
requirements.txt      — Extra pip dependencies
prompts/
  system.txt          — System prompt for the LLM
tools/
  __init__.py         — Custom function tools
webhooks/
  __init__.py         — Webhook handlers
pipelines/
  __init__.py         — Custom pipeline stages
utils/
  __init__.py         — Shared helpers

AI Coding Assistant

Every custom agent includes an AI coding assistant powered by Gemini. Instead of writing code manually, describe what you want in plain English: Example prompts:
Create a voice agent that can search the web using Tavily to answer questions about current events, news, or facts. Always use the Tavily days parameter set to 7 so results are only from the last week. Keep responses conversational.
Add a tool that checks order status by calling our REST API at https://api.example.com/orders
Switch the STT provider to Deepgram and add sentiment analysis
The assistant will:
  • Generate all file changes
  • Detect when API keys are needed and show a secure input form
  • Pick an appropriate voice for the agent
  • Follow LiveKit Agents SDK best practices

Environment Variables

Custom agents can use third-party API keys securely:
  • Platform keys (LiveKit, Google, Resemble, STT) are injected automatically from your dashboard Settings
  • Custom keys (e.g., Tavily, Stripe, your own APIs) are added per-agent via the Environment Variables tab or through the AI assistant
All values are encrypted with AES-256 in the database and injected into the container at runtime. They never appear in code or logs.

Preview vs Deploy

PreviewDeploy
PurposeTest in browserProduction calls
ContainerTemporary, connects to one roomPersistent worker
TelephonyBrowser-only (WebRTC)Inbound + outbound phone calls
LifecycleStops when you close previewRuns until stopped, auto-restarts

Preview

Click Preview to start a browser-based voice session. A temporary container spins up, connects to a LiveKit room, and you can talk to your agent immediately. The container is removed when the preview ends.

Deploy

Click Deploy to start a persistent worker container. The worker:
  • Registers with LiveKit as arkenos-custom-{agent_id}
  • Accepts dispatched calls (inbound SIP and outbound)
  • Runs with restart: unless-stopped (auto-recovery)
  • Can be stopped from the dashboard

Writing Tools

Tools let your agent perform actions during a conversation:
from livekit.agents import function_tool, RunContext

@function_tool
async def search_web(query: str, context: RunContext) -> str:
    """Search the web for current information."""
    async with httpx.AsyncClient() as client:
        resp = await client.post("https://api.tavily.com/search", json={
            "api_key": os.environ.get("TAVILY_API_KEY"),
            "query": query,
            "days": 7,
            "max_results": 3,
        })
        results = resp.json().get("results", [])
        return "\n".join(f"- {r['title']}: {r['content'][:200]}" for r in results)
Add tools to your agent:
class MyAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions="You are a helpful assistant with web search.",
            tools=[search_web],
        )

Speak-while-executing

For slow tools, say a filler phrase so the caller isn’t left in silence:
@function_tool
async def slow_lookup(query: str, context: RunContext) -> str:
    """Look up information."""
    await context.session.say("Let me look that up, one moment.")
    # ... do the slow thing
    return result

File Versioning

Every file edit is versioned. You can:
  • View version history for any file
  • Roll back to a previous version
  • See what changed between versions

Build and Image Caching

When you build a custom agent:
  1. All files are downloaded from MinIO
  2. A Docker image is built on top of arkenos-agent-base:latest
  3. The image is tagged with a hash of requirements.txt
  4. If requirements haven’t changed, the cached image is reused
The base image pre-installs all LiveKit plugins, common libraries, and the Silero VAD model — so your custom builds are fast.

Pre-Installed Libraries

Available without adding to requirements.txt:
CategoryLibraries
LiveKitlivekit-agents, all plugins (assemblyai, deepgram, google, silero, elevenlabs, resemble)
HTTPhttpx, aiohttp
Datapydantic, numpy, pandas
Loggingusage_logger (cost tracking)