audio-transcriber-memory — Cross-session meeting transcription with voiceprint memory

English | 中文

Transcribe a meeting once, remember who spoke — and keep that memory across every session that follows. This is an enhanced fork of the audio-transcribe skill (v1.9.1), built for repeated, cross-session meeting transcription: enroll a speaker's voiceprint once, and every later recording is auto-matched so the same person keeps the same label, session after session.

Why "memory" changes everything

The upstream skill labels speakers by first-appearance order in each recording — accurate within a single session, but meaningless across sessions. Speaker 1 in Monday's meeting and Speaker 1 in Friday's meeting may be two different people, and any labeling work you did in between is lost.

This fork adds a complete enroll → match → merge loop:

Session A (enroll)                       Session B (match)
  ┌──────────────────────┐                ┌──────────────────────┐
  │ enroll_speakers.py   │  speaker-      │ transcribe.py        │
  │ → CAM++ voiceprint   │  embeddings    │ → Hungarian 1:1 match│
  │ → L2 centroid vector │ ─────────────► │ → threshold filter   │
  │ → gender detection   │     json       │ → new-speaker fallback
  └──────────────────────┘                └──────────────────────┘

• enroll_speakers.py — extract CAM++ voiceprint centroids from a well-labeled transcript
• transcribe.py Phase 2 — match the voiceprint DB against new clusters with the Hungarian algorithm (optimal one-to-one matching, default similarity threshold 0.5), eliminating swapped labels
• --merge — weight-average new voiceprints back into the project DB automatically, with no hand-editing of JSON

Key enhancements

What's inside

The skill follows a 6-step SOP — project-context detection, environment setup, supporting files, engine selection & transcription, verification, and voiceprint-DB updates — plus a fast path that skips information-gathering when the project context is already clear.

★ Default choice. auto and whisper lack per-sentence timestamps, so speaker diarization is unavailable. Use zh, zh-basic, en, or mimo when you need speaker labels.

Requirements

• Python 3.12+
• ffmpeg
• FunASR (auto-installed by setup_env.sh)
• CUDA GPU recommended (CPU also works)
• MiMo-V2.5-ASR engine needs CUDA ≥ 12.0 + ≥ 20 GB VRAM (optional)

Quick start

# 1. Environment setup
SCRIPTS=./skills/audio-transcribe/scripts
AUTO_YES=1 bash $SCRIPTS/setup_env.sh

# 2. Chinese meeting transcription (local only, no data upload)
python3 $SCRIPTS/transcribe.py meeting.m4a \
    --lang zh --num-speakers N --speakers "Name1,Name2,..." \
    --hotwords hotwords.txt

# 3. Use the project voiceprint DB (auto-match known speakers)
python3 $SCRIPTS/transcribe.py meeting.m4a \
    --lang zh --speakers "Name1,Name2,..." \
    --speaker-embeddings project-speaker-embeddings.json

# 4. Enroll voiceprints from a confirmed transcript
python3 $SCRIPTS/enroll_speakers.py transcript_raw_transcript.json audio.flac \
    --speakers "Name1,Name2,..." --output speaker-embeddings.json

# 5. Merge into the project DB
python3 $SCRIPTS/enroll_speakers.py transcript_raw_transcript.json audio.flac \
    --speakers "Name1,Name2,NewPerson" \
    --output project-speaker-embeddings.json \
    --merge project-speaker-embeddings.json

Privacy: transcription is fully local by default. Adding --model <id> enables optional LLM cleanup (AWS Bedrock / Anthropic / OpenAI-compatible) — this sends transcript excerpts to the provider. Omit --model to keep all data on your machine.

Documentation

License

This project inherits the license of the upstream zxkane/audio-transcriber project.

Acknowledgments

Built on top of zxkane/audio-transcriber's audio-transcribe skill — keeping its FunASR / MiMo ASR engines, VAD + CAM++ speaker diarization, and LLM cleanup. Thanks to the original author for the excellent foundation.

⬆ Back to top