Kai

A pocket voice pal for your self-hosted AI agent. Kai gives a Hermes Agent
a body you can clip to your bag: push-to-talk voice, a pixel face, a badge and a chime, on a 20 g ESP32 stick.

_{Kai on a real M5StickS3 (48 × 24 × 15 mm, 20 g), clipped to a lanyard.}

Hold the button on the M5StickS3 and ask. Kai answers out loud, with a pixel face and a small info
card:

• Quick questions get an answer in about two seconds, from Gemini Live.
• Bigger jobs ("research…", "plan…", "tell me later") go to your agent, and Kai tells you when they're
  done.
• Your agent remembers you across conversations, and can wake the Stick for reminders, briefings and
  watches.

How it works: a fast path and a slow path

                      home server
M5StickS3 ──TLS────▶ kai-relay ──▶ Gemini Live ................ FAST PATH: answers in ~2 s
 push-to-talk  Opus │   │          + Google Search, Kai's tools   (tides, weather, light, parking, notes)
 face + cards       │   │
                    │   └─ ask_agent ──▶ Hermes Agent ......... SLOW PATH: seconds to minutes
                    │                     (Docker, locked down)   (research, planning, many searches)
                    └──── inbox ◀── results, and Hermes's own updates (kai_notify)

• The Stick stays thin. It streams Opus audio (~24 kbit/s up, ~32 kbit/s down) over home Wi-Fi,
  draws the face and cards, and deep-sleeps between uses. The link to the relay is encrypted and
  mutually authenticated (TLS-PSK, one key per device).
• The relay (Python, on any always-on Linux box) bridges the Stick to Gemini Live. It also:
  • runs Kai's own tools: NOAA tides, Open-Meteo weather and marine, golden and blue hour, Google
    Places parking, trip notes;
  • makes sure a card appears even when Gemini skips a tool;
  • hands slow work to Hermes, and keeps the inbox.
• Hermes runs next to the relay in a pinned Docker container: web search, memory and skills, but no
  shell, file or browser tools, and nothing acts unattended. After each conversation it updates its
  memory of you and sends Kai a short brief, so Kai starts every conversation already knowing you.
  It reaches back into Kai through a small MCP server on localhost (kai_notify and Kai's data tools).
  Anything it brings back from the web is treated as data, never as instructions.

For Hermes Agent and OpenClaw users

Your agent already researches, remembers and runs on a schedule. Kai is the device that lets you talk to
it, and hear back from it, away from a keyboard.

• Hermes Agent works today: a pinned, locked-down Docker setup in
  relay/deploy/hermes/, applied by one configure.sh. There's no shell,
  file or browser access, nothing acts unattended, and anything it brings back from the web is treated
  as data, never as instructions.

• The contract is small, so other agents can plug in. The relay asks the agent; the agent answers
  through Kai's MCP server:

  Direction	What	Hermes today
  relay → agent	start a background run with a task; poll for {speak, card, details_md}	POST /v1/runs (relay/kai_relay/hermes.py, under 100 lines)
  agent → Kai	kai_notify: a spoken update now, or a badge if the Stick is asleep	MCP client, bearer token, localhost
  agent → Kai	kai_profile_brief: the short memory brief Kai's voice starts with	MCP, with a one-time key per memory run
  agent → Kai	read-only data tools: tides, weather, sun and light	MCP
  agent schedule	briefings and watches report through kai_notify; the relay wakes the Stick for them	Hermes cron jobs

• OpenClaw has the same pieces (background work, cron, a heartbeat), but there's no adapter yet:
  one would replace hermes.py and call Kai's MCP tools. Contributions are welcome.

Screens

Weather, mid-answer	Tides	Saved note

_{Rendered from the firmware's sprite and layout code (firmware/src/face.cpp) at 3x; fonts are approximate.}

What you can ask

"Here" and "near me" mean the home location in the relay's .env (KAI_HOME_*).

Using it

• Badge: a yellow number in the top-left means updates are waiting.
• Chime: two rising notes mean Kai is about to tell you something you didn't just ask: a reminder,
  a briefing, a watch that fired, or a finished task. The Stick wakes itself for these; watches stay
  quiet at night (22:00–07:00).
• Sleep: Kai dims after 15 s. On battery it deep-sleeps after 45 s; either button wakes it, and
  holding the front button to wake records at once, so you can just press and talk. On USB the screen
  turns off at 45 s but Kai stays connected.
• Battery: about 6 days per charge at 20 questions a day (projected, see
  docs/POWER.md).

Setup

1. Relay on an always-on Linux box on your LAN (Raspberry Pi, mini PC, NAS); no sudo needed.

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/zbruceli/kai ~/kai && cd ~/kai/relay
cp .env.example .env && chmod 600 .env && nano .env   # GEMINI_API_KEY, KAI_DEVICE_TOKEN, KAI_HOME_*
uv run kai-psk kai-stick1                             # a device key: KAI_DEVICE_PSKS line into .env
uv sync --frozen && uv run kai-relay                  # prints wss://<server-ip>:8443/ws

• Keys: get a Gemini key from AI Studio. Parking also needs a
  Maps key with Places API (New); everything else works without it.
• Run at boot: copy deploy/kai-relay.service to ~/.config/systemd/user/, set its TZ=, then
  loginctl enable-linger "$USER" and systemctl --user enable --now kai-relay.
• Logs: journalctl --user -u kai-relay -f shows the transcript and tool calls.
• Update: git pull && uv sync --frozen && systemctl --user restart kai-relay.
• Fixed IP: give the server a DHCP reservation, because the Stick connects to its IP.

2. Slow path (optional): Hermes in Docker. Follow relay/deploy/hermes/README.md:
docker compose up -d, a few secrets, then ./configure.sh. Without it, Kai simply has no ask_agent.

3. Stick.

cd firmware && cp src/secrets.example.h src/secrets.h   # Wi-Fi, RELAY_HOST, token, the kai-psk lines
pio run -t upload

If the port isn't found, hold the side button ~2 s until the green LED blinks (download mode). Builds
embed your Wi-Fi password, token and device key, so releases are source only; never share a built
firmware.bin. The link to the relay is encrypted and mutually authenticated (TLS-PSK); see
docs/ARCHITECTURE.md.

Without a Stick: uv sync --extra sim && uv run kai-sim --host <server-ip> talks to the relay through
the Mac's mic and speakers. Give it its own key: KAI_SIM_PSK=kai-sim:<hex> in .env, with the same entry
in the relay's KAI_DEVICE_PSKS.

Tests: cd relay && uv run pytest (offline). Add -m network -s to hit NOAA and Open-Meteo for real.
uv run python evals/routing.py checks which path Gemini Live picks for a set of requests (real Gemini, ~1 min per 30 cases).

Repo map

firmware/src/   main.cpp (modes, buttons, Wi-Fi, sleep) · audio.* (mic, speaker) · voice_codec.* (Opus) · face.* (sprite, cards)
firmware/scripts/patch_websockets.py   adds TLS-PSK to the WebSockets library at build time
relay/kai_relay/
  server.py     listeners (wss:// with TLS-PSK) and device auth
  tls.py        the encrypted link; `kai-psk` makes device keys
  session.py    Stick <-> Gemini Live bridge; wire protocol at the top
  tools/        fast-path tools, one file each (@tool decorator)
  backstop.py   runs the tool itself when Gemini answers without one, so a card always appears
  agent.py      slow path: hand-off to Hermes, the inbox, the badge
  hermes.py     Hermes API client
  schedule.py   reminders, briefings, watches, and when the Stick should wake
  mcp_server.py Kai MCP server that Hermes calls back into
relay/evals/    routing eval: which path Gemini Live picks for a set of requests
relay/deploy/   systemd unit · hermes/ (pinned compose, configure.sh, SOUL.md)

More in docs/ARCHITECTURE.md (protocol, tools, security),
docs/POWER.md and CHANGELOG.md.

Roadmap

• Voice, search, tides, weather, light, parking, notes, with cards
• Deep sleep with press-and-talk from sleep; Opus audio; power savings
• Slow path, phase 1: hand-off to Hermes, inbox with badge, results spoken when ready
• Slow path, phase 2: memory across conversations (transcripts to Hermes, a profile brief Kai starts with, recall)
• Slow path, phase 3: reminders, briefings and watches that wake the Stick on a timer
• Encrypted Stick link (TLS-PSK), security audit and hardening, routing eval
• Approvals: a spoken read-back plus a long press before anything with side effects
• Remote access: phone as a BLE bridge (GPS for "near me"), or a relay reachable away from home
• IMU gestures (lift to wake, shake to hush), OTA updates

License

MIT, see LICENSE.