livetalk
Health Warn
- License — License: MIT
- Description — Repository has a description
- Active repo — Last push 0 days ago
- Low visibility — Only 8 GitHub stars
Code Pass
- Code scan — Scanned 5 files during light audit, no dangerous patterns found
Permissions Pass
- Permissions — No dangerous permissions requested
No AI report is available for this listing yet.
Realtime duplex voice chat with your Hermes agent — a composer mini-bar plugin. Gemini Live today, more voice providers tomorrow.

Livetalk
Talk to it. It talks back — and when real work is needed, the Hermes agent does it.
✨ See the face-ghost move (live demo)
What's Hermes? Hermes Agent is the open-source AI agent by Nous Research that lives in your desktop app — it chats, runs tools, browses the web, remembers things, and automates your work. This plugin gives it a voice you can talk to.
A composer mini-bar for the Hermes desktop app: a full-duplex voice session in one small popover. Small talk is answered by the voice model itself; anything that needs facts, your machine, or tools is handed to a dedicated Hermes agent session, and the answer is spoken back to you.
The renderer never talks to Google or ElevenLabs. A local Python backend owns the upstream WebSocket and your API key; the desktop UI only exchanges data with
localhost.
Two voice providers
| Gemini Live (Google) | ElevenLabs Realtime | |
|---|---|---|
| Models | gemini-3.8-live, gemini-2.5-flash-native-audio-latest, …-preview-12-2025, …-preview-09-2025, gemini-3.1-flash-live-preview |
eleven_realtime_v1_mini, eleven_realtime_v1_max |
| Voices | 30 HD prebuilt (Puck, Aoede, Kore, Leda, …) | every voice on your account (17k+ library) |
| Languages | any documented BCP-47 pair (primary + secondary) | auto-detected, 90+ languages incl. Indonesian & Javanese |
| Transcript | streaming, word by word | streaming; expressive tags ([calm], [laughs]) stripped so the text matches the speech |
| Key | GOOGLE_API_KEY |
ELEVENLABS_API_KEY |
Pick the provider in Settings → Plugins → livetalk. Each provider keeps its own model/voice/language fields — switching never mixes them. Tools (talk, recall), barge-in, transcript and the token meter behave the same on both.
What you get
- Voice ↔ Hermes delegation — greetings and general knowledge are answered by the voice model; anything needing live data, your machine, or tools is delegated via a
talkcall to a dedicated "Livetalk" agent session, and the agent's final answer is spoken aloud. - Session memory (
recall) — a second tool lets the voice model search earlier messages of the very same Hermes session, so "what did we talk about yesterday?" is answered from real history instead of a guess. - Warm start — every session opens with the last five messages plus a compact topic digest (your recent requests and what came of them), so continuity survives app restarts.
- One persistent session — the agent session id is remembered (
ctx.storage) and matched by title, so the conversation is never re-created behind your back. - Full-duplex + barge-in — talk over the model mid-sentence. The client cuts its playback queue immediately; the server-side interruption path covers the mid-generation case.
- Liquid blob indicator — one animated blob instead of text: violet and swelling with your voice while listening, green while the model speaks, spinning teal while the agent works. Click it to start/stop.
- Live transcript — both sides rendered as flowing sentences (streaming fragments merged per speaker). Toggleable.
- Session token meter — real
usageMetadatafrom Google, shown as12.3k tok, reset per session. - Session resumption + auto-redial — a network drop reconnects with the resumption handle, keeping context. A manual stop never reconnects.
- Secure by design — API keys live in
~/.hermes/.envand never leave the backend process. The UI only receives short-lived session tokens.
Requirements
| Hermes | Desktop app with plugin support; tested on v0.23.3 (requires_hermes: >=0.22.0). The settings page needs a build with registerSettingsPage (~0.23+). |
| API key | One of: GOOGLE_API_KEY (AI Studio, Live API access) or ELEVENLABS_API_KEY (elevenlabs.io, Realtime Conversations) — in ~/.hermes/.env |
| Python | Nothing extra — the backend uses only Hermes-bundled packages (fastapi, websockets) |
| Hardware | Microphone + speakers/headphones (headphones strongly recommended — see Tips) |
| OS | Windows / macOS / Linux desktop |
Install
hermes plugins install cruzlxyz/livetalk
This installs both halves (the Python dashboard backend and the desktop mini-bar), then restart the desktop app once.
One-time setup — put at least one key in ~/.hermes/.env:
GOOGLE_API_KEY=AIza...
ELEVENLABS_API_KEY=sk_...
Then click the blob in the status bar and just talk.
Usage
- Click the blob to start; click again to stop.
- Just talk. Examples of how turns route:
| You say | What happens |
|---|---|
| "What is Hermes anyway?" | The voice model answers itself — general knowledge |
| "What's the BTC price right now?" | Delegated (talk) — the agent searches the live web, the answer is spoken |
| "Check how many files are in my Downloads folder" | Delegated — the agent runs tools on your machine |
| "What did we talk about yesterday?" | recall — searches this session's earlier messages |
| "Wait, how many percent did you say?" | Answered from context — no re-search |
- Blob colors: violet = listening · green = speaking · spinning teal = Hermes is on it · gray = idle.
- Use headphones: without them the microphone re-captures the model's voice, which is the #1 cause of chaotic replies in every voice app.
- Google caps Live audio sessions (~15 min); the backend auto-resumes on network drops.
Settings (Settings → Plugins → livetalk)
One page, provider-aware — only the active provider's fields are shown.
| Setting | Scope | Values | Default |
|---|---|---|---|
| Provider | common | gemini / elevenlabs |
gemini |
| Live model | Gemini | the five Live models above | gemini-3.8-live |
| Voice | Gemini | 30 HD prebuilt voices | Puck |
| Primary language | Gemini | any documented BCP-47 code | id-ID |
| Second language | Gemini | none or any BCP-47 code (code-switching) |
en-US |
| ElevenLabs model | ElevenLabs | eleven_realtime_v1_mini / _max |
_mini |
| ElevenLabs voice | ElevenLabs | every voice on your account | brian |
| ElevenLabs language | ElevenLabs | auto or a fixed language |
auto |
| Smart transcription | common | on/off — removes fillers, fixes false starts | on |
| Spoken answers | common | full / brief (2–4 sentences) |
full |
| Show transcript | common | show the popover conversation text | on |
| Show diagnostic log | common | show the timestamped event log | on |
What this plugin touches
- Dashboard API (
dashboard/plugin_api.py): FastAPI routes under/api/plugins/livetalk/*— session proxy for both providers, delegation + recall queues, settings read/write. - Desktop surface (
desktop/plugin.js): one composer blob + popover (blob control, token counter, transcript, log), mic capture, playback queue, delegation worker. - One agent session: creates and reuses a Hermes session titled "Livetalk" for all delegations. No agent tools, no hooks, no middleware, no core overrides.
API routes
| Method | Route | Purpose |
|---|---|---|
GET |
/config |
{ model, candidates[], show_transcript, show_log } — model catalog + toggles |
GET |
/settings |
Settings page data: { values, fields } for the active provider |
POST |
/settings |
Persist settings (per-provider keys kept apart) |
POST |
/start |
Dial the active provider → live session. Returns { ok, model, provider, resumed, speak } |
POST |
/audio |
Forward one mic chunk upstream ({ data: <base64 PCM16 mono 16kHz> }) |
GET |
/poll |
Incremental state: transcriptions, audio, delegations, recalls, usage, barge-in counter — all offset-based |
POST |
/tool_response |
Answer a queued talk delegation or a recall lookup |
POST |
/close |
Tear down the session (blocks auto-redial) |
GET |
/status |
Liveness probe |
How it works
Three flows, one router — the voice model decides per turn:
- Small talk / general knowledge → answered directly over the audio channel.
- Real work → the model emits
talk(request)→ the renderer submits it to the dedicated agent session (prompt.submit, surfacevoice-live, with recent spoken context) → polls the submission receipt until it settles → the agent'sfinal_responseis fed back as the tool response → spoken aloud. - Session memory → the model emits
recall(query)→ the renderer searches this session's history → the matching messages are returned as the tool response → answered from them.
Full details and diagrams: docs/WORKFLOW.md.
┌─ Hermes desktop (Electron renderer) ────────────┐
│ composer blob + popover (plugin.js) │
│ mic capture (WebAudio PCM16/16k) │
│ playback + output level analyser (24k) │
│ delegation queue + recall lookup │
└──────────────┬──────────────────────────────────┘
│ ctx.rest() → localhost only
┌──────────────▼──────────────────────────────────┐
│ Backend (plugin_api.py, FastAPI router) │
│ • owns the upstream WebSocket (either provider)│
│ • Gemini: mints 1-use ephemeral tokens │
│ • ElevenLabs: mints a realtime client session │
│ • queues talk + recall calls │
│ • reads/writes the plugin settings page │
└───────┬────────────────────────┬────────────────┘
│ wss://… │ prompt.submit / prompt.receipt
▼ ▼
Gemini Live / ElevenLabs Hermes agent session
Realtime (v1alpha) "Livetalk" (full toolset)
Compatibility notes
- Settings page needs a Hermes build with plugin settings pages (~0.23+). Older builds: everything works, the page just does not appear.
gemini-3.8-live-extended-thinkingis deliberately not offered — verified live, that model never emits tool calls, so it cannot delegate or recall. The picker only lists models that can do real work.- Transcription language hints follow the Live API transcription docs (
languageCodes, SMART mode, custom vocabulary). - The backend reads the plugin settings store through two Hermes-internal helpers (
_plugin_settings_entry,save_plugin_setting) because no public seam exists yet; every call is wrapped, so a future rename degrades to defaults instead of breaking a session.
Privacy
Transcripts and audio exist only in memory while a session runs; the plugin writes no transcripts to disk. Your API keys never leave the backend process. On the provider side, the key owner's data-use policy applies.
License
MIT — see LICENSE.
Reviews (0)
Sign in to leave a review.
Leave a reviewNo results found