OpenLive: Give Your AI Coding Agent a Voice That Runs Locally

By Prahlad Menon 2 min read

You’re deep in a coding session with Claude Code. Your hands are on the keyboard, but you wish you could just talk to it — explain what you want, hear it think out loud, interrupt when it’s going the wrong direction.

OpenLive makes that work. It’s an open-source voice and vision layer that runs entirely on your machine. The speech recognition, text-to-speech, and voice activity detection all happen locally via WebGPU. No audio uploads, no per-minute fees like ElevenLabs or OpenAI Realtime.

What It Actually Does

OpenLive sits between you and your AI:

mic → VAD → STT → your AI → TTS → speaker
     (local) (local)  ↑    (local)
                      └── Claude Code, Codex, Cursor, 
                          or any model with an API key

The voice pipeline runs in your browser using WebGPU:

  • Silero VAD — detects when you’re speaking
  • Whisper STT — transcribes your speech to text
  • Smart-Turn — knows when you’ve actually finished talking (not just paused)
  • Kokoro/Supertonic TTS — speaks the AI’s response back to you

The only thing that leaves your machine is the text transcript (and optionally camera/screen frames for vision). Your voice audio never uploads anywhere.

Why This Matters: The Cost Comparison

ServiceWhat You Pay
ElevenLabs AgentsPer-minute audio fees + model costs
OpenAI RealtimePer-minute audio fees + model costs
Gemini LiveGoogle’s pricing
OpenLiveJust your normal model API costs (or $0 with local agents)

If you’re already paying for Claude Code or have a Codex subscription, OpenLive adds voice for free. If you’re using Ollama locally, the entire stack costs nothing.

Quick Start: Get Talking in 5 Minutes

Option 1: Download the app

  1. Grab the installer from GitHub releases (macOS, Windows, Linux)
  2. Open the app
  3. Either paste an API key (OpenAI, Anthropic, etc.) OR pick your coding agent in Settings → Agents
  4. Start a call

First run downloads ~200MB of voice models from Hugging Face. After that, it’s cached.

Option 2: Run from source

git clone https://github.com/katipally/openlive.git
cd openlive
pnpm install
pnpm desktop:dev

Requires Node.js 22.13+ and pnpm.

Practical Use Case 1: Voice-Drive Claude Code

This is the killer feature. Connect OpenLive to Claude Code and you can:

  • Describe what you want verbally — “Refactor this function to use async/await”
  • Hear the agent’s plan — It speaks its step-by-step thinking
  • Interrupt anytime — Say something and it stops mid-sentence (barge-in)
  • Answer permission prompts by voice — “Yes” / “No” when it asks to run commands

Setup:

  1. Open Settings → Agents
  2. Click Install next to Claude Code (or Codex, Cursor, OpenCode)
  3. Sign in when prompted
  4. Start a call, pick the agent, select your project folder

Your conversation lands in ~/.claude/projects/... — the same place Claude Code’s CLI stores sessions. You can resume from either the CLI or OpenLive.

Practical Use Case 2: Hands-Free Debugging

You’re looking at a stack trace. Instead of typing:

“This error is happening in the payment module. The stack trace shows it’s failing at line 47 of processor.ts. I think it’s a null reference but I’m not sure why user.paymentMethod would be null here.”

Just… say that. Out loud. While looking at the code.

OpenLive can also share your screen or camera with vision-capable models, so you can say “look at this error” and it sees what you see.

Practical Use Case 3: Clone Your Own Voice

Settings → Clone Voice lets you record 5-30 seconds of yourself. From then on, the AI speaks in your voice.

This runs locally using ZipVoice (Apache 2.0). The model is ~208MB, downloaded on first use, and you can delete it anytime. Profiles export/import between machines.

Important: Only clone your own voice or one you have permission to use.

Practical Use Case 4: Mini Mode While You Work

Shrink OpenLive to an always-on-top pill. It keeps listening while you code in your editor. When you need something, just talk. Notifications and a menu-bar tray keep you connected without a full window.

What Models Work?

With an API key:

  • Anthropic (Claude)
  • OpenAI (GPT-4, etc.)
  • Google (Gemini)
  • xAI (Grok)
  • DeepSeek
  • Groq
  • Ollama (fully local — no API costs at all)

As a coding agent (via ACP):

  • Claude Code
  • Codex
  • Cursor
  • OpenCode
  • Hermes

The Agent Client Protocol (ACP) is a JSON-RPC standard for talking to coding agents. OpenLive speaks it natively.

Architecture: How It Stays Private

┌──────────────────── your machine ────────────────────┐
│  renderer (browser)              agent server        │
│                                                      │
│  mic → VAD → STT → text ─┐  WS  ┌─ your coding agent│
│                          ├─────▶│  (local stdio)    │
│  speaker ← TTS ← reply ──┘◀─────┘                   │
│                                                      │
└──────────────────────────────────────────────────────┘
  • Audio processing happens in the browser renderer via WebGPU
  • The WebSocket carries only text + optional image frames
  • API keys are encrypted at rest (AES-256-GCM)
  • Only the last 4 digits of keys are ever displayed

Limitations (Honest Assessment)

It’s not GPT-4o voice mode. OpenLive is a cascaded pipeline (speech → text → model → text → speech), not a native speech-to-speech model. That means:

  • Slight latency between when you stop talking and when the AI starts responding
  • The AI can’t overlap listening and talking the way GPT-4o can
  • Emotional nuance in your voice doesn’t transfer to the model

WebGPU required. Modern Chrome, Edge, or Firefox. Safari support is limited.

First-run download. ~200MB for the voice models. Slow on bad connections.

Node 22.13+ required for building from source.

FAQ

Does my voice audio get uploaded anywhere?

No. Speech recognition runs locally via WebGPU. The only thing sent to the model is the text transcript.

Can I use this without any cloud APIs?

Yes. Use Ollama as your backend and everything runs locally. Zero cost, zero data leaving your machine.

Does it work with my existing Claude Code sessions?

Yes. Sessions are stored in the same location Claude Code uses. You can start in OpenLive and resume in the CLI, or vice versa.

How much does it cost?

OpenLive itself is free and open source (MIT). You pay only your normal model API costs — no audio surcharges.

Can I use this on a laptop without a GPU?

WebGPU will use whatever GPU is available. On integrated graphics, expect higher latency. CPU-only fallback is limited.

What about Linux?

Supported. Download the AppImage or .deb from releases, or build from source.

When to Use OpenLive

✅ Good fit:

  • You want to talk to your coding agent hands-free
  • You’re already paying for Claude Code / Codex and want voice without extra fees
  • Privacy matters — you don’t want audio uploaded
  • You want to clone your own voice for a personalized assistant

❌ Not the right tool:

  • You need real-time speech-to-speech with overlapping talk (use GPT-4o voice)
  • You’re looking for a phone/call-center solution
  • You need production-grade reliability (it’s still early-stage OSS)

If you’ve ever wished you could just talk to Claude Code while keeping your hands on the keyboard, OpenLive is worth the 5-minute install. The local-first architecture means no surprise audio bills, and the coding agent integration is genuinely useful — not just a demo.