audio-production-skill

agent
Security Audit
Warn
Health Warn
  • License — License: MIT
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Low visibility — Only 6 GitHub stars
Code Pass
  • Code scan — Scanned 12 files during light audit, no dangerous patterns found
Permissions Pass
  • Permissions — No dangerous permissions requested

No AI report is available for this listing yet.

SUMMARY

Agent skill that deterministically executes typed audio operation graphs via ffmpeg-skill.

README.md

audio-production-skill app icon

audio-production-skill

Deterministic audio production / processing Skill for the AI Video Production Ecosystem
Executes a typed operation graph through ffmpeg-skill, with validated artifacts and provenance out.
NOT an AI agent: no LLM, no prompt, no reasoning, no decision.

CLI · Supported operations · Architecture · Contract

tests Python 3.9+ MIT License

Deterministic audio production / processing Skill for the AI Video Production Ecosystem: gain, sample-accurate
trim / cut, silence removal (explicit ranges), fades, EBU R128 loudness normalisation, mix, concat, mono / stereo /
surround down-mix, FFT noise reduction, typed dynamics (gate / compressor / limiter), format conversion and audio
extraction from video containers, executed as a typed operation graph through
ffmpeg-skill, with validated artifacts and provenance out.

audio-production-skill is NOT an AI agent. It contains no LLM, no prompt, no reasoning, no decision, no
production plan. It executes what a typed request says, refuses everything else, and reports what it observed.

audio-production skill --json              # Skill / Capability / Tool contract (alias: contract --json)
audio-production doctor --json             # environment vs. contract: ffmpeg-skill, ffmpeg, capabilities per operation
audio-production validate - --json         # validate a request document, run nothing, read no media
audio-production plan - --json             # dry run: graph, tool selection, expected timeline; writes no media
audio-production run - --json              # execute (stdin: request document; stdout: exactly one response document)

Requirements: Python 3.9+, standard library only; an ffmpeg-skill checkout (any version: there is no version
window — compatibility is detected from the checkout's own contract, which must declare probe, audio, cut,
loudness and join with every flag this skill emits; optional flags such as --overwrite are used only where
declared, see docs/ffmpeg-skill.md; CI runs against both commit 336e0c4 (0.12.2) and ffmpeg-skill main) and
FFmpeg (ffmpeg + ffprobe) on PATH for ffmpeg-skill. Install: pip install -e .

What it is, and what it is not

ffmpeg-skill media-analysis-skill audio-production-skill video-production-agent
Role low-level media execution engine (hands) measurement / observation (meters) audio production execution (typed audio operations) reasoning / decision / planning / orchestration (brain)
Does runs ffmpeg for cut, audio post, loudness, join, export, … measures loudness, silence, streams, integrity executes GAIN / TRIM / CUT / SILENCE_REMOVE / FILLER_REMOVE / FADE / NORMALIZE / LOUDNESS_SET / MIX (with ducking) / STEMS / CONCAT / MONO / STEREO / DOWNMIX / NOISE_REDUCTION / DYNAMICS / VOICE_CLEANUP as a dependency graph, validates every artifact, records provenance decides whether and what to process, builds the request
Never holds a project model edits or writes media measures for decisions, decides which ranges are silence, recognises speech, invents parameters, runs ffmpeg directly, accepts commands / filters runs ffmpeg
  • audio-production-skill ≠ media-analysis-skill. media-analysis-skill says "silence from 0.0 to 2.0 s, −23.1 LUFS".
    audio-production-skill only executes "remove [0.0, 2.0)", "normalise to −16 LUFS / −1.5 dBTP". The one measurement
    this skill performs is the verification of its own NORMALIZE and LOUDNESS_SET output (re-measured through ffmpeg-skill/loudness);
    it never reports a measurement of an input for a decision.
  • audio-production-skill ≠ ffmpeg-skill. ffmpeg-skill runs FFmpeg per script call. audio-production-skill owns the
    audio project model: sources, tracks, operation graph with deterministic identities, source ↔ timeline mapping,
    intermediate management, output validation and provenance. It never calls ffmpeg itself: every process it starts
    is python3 <ffmpeg-skill>/scripts/{probe,audio,cut,loudness,join}.py with a typed argv.
  • Loudness: owner and engine. This repository owns 音量・ラウドネス; ffmpeg-skill is the engine. NORMALIZE and
    LOUDNESS_SET run through ffmpeg-skill loudness. The
    agent's video path keeps calling ffmpeg-skill/loudness directly; that routing changes only after a measured comparison
    of loudnorm and ebur128 on the same files (video-production-agent step 34). That comparison is committed
    (ADR-116, #160): on the same fixtures both paths write the same samples, on target by both meters, so the
    routing stays as it is. See the
    Velisa ownership decisions 2026-11-12 (D23).
  • audio-production-skill ≠ video-production-agent. There is no Observation → Inference → Decision → ProductionPlan
    here. The agent decides; this skill executes one typed request and says exactly what happened.

Architecture

external caller (video-production-agent adapter)
   │  JSON request  audio-production/request@1   (stdin)
   ▼
audio-production run - --json
   ├─ model.parse_request        typed validation: schema, ids, references, parameter schemas per operation type,
   │                             forbidden fields (command / argv / filter / …), formats, layouts, sample rates
   ├─ security.PathPolicy        inputs are regular files (symlinks resolved, optional allowed roots);
   │                             outputs inside the workspace, never an input, never an existing file unless overwrite
   ├─ graph.OperationGraph       nodes (tracks, implicit track range / gain, operations), deterministic topological
   │                             order, cycle / unreachable detection
   ├─ adapter.FfmpegSkill.probe  every source probed (read-only) and sha256-fingerprinted; a video container's audio
   │                             track is extracted to a PCM WAV intermediate (ffmpeg-skill/audio)
   ├─ graph.identities           operation_id = sha256(type, parameters, input identities, tool versions)
   ├─ plan                       tool per node, argv template, expected timeline (segments) and duration
   ├─ execute (in order)         ffmpeg-skill/{audio,cut,loudness,join} → PCM WAV intermediate, reused when the identity
   │                             and input hashes match a manifest; sidecars and partial files removed on failure
   ├─ validate                   exists, size > 0, readable, audio stream, codec, duration ± tolerance, channels,
   │                             sample rate, sha256; NORMALIZE re-measured (loudness / true peak vs. tolerance)
   ├─ export outputs             ffmpeg-skill/audio to the requested format, validated against `expect`
   ▼
JSON response  audio-production/response@1   (stdout, exactly one document; stderr = diagnostics)

Full description: docs/architecture.md.

Skill / Capability / Tool

  • Skill: audio-production (this package), kind execution, one tool audio-production/run.
  • Capabilities (vocabulary of video-production-agent's CapabilityResolver): ffmpeg-skill, ffmpeg, ffprobe,
    filter:<name>, encoder:<name>. doctor reports each as supported, unsupported or unknown (core ffmpeg
    filters are not probed by ffmpeg-skill's doctor and are therefore reported unknown, then verified per run).
  • Tools used (all through ffmpeg-skill's public contract 1.0): ffmpeg-skill/probe, ffmpeg-skill/audio,
    ffmpeg-skill/cut, ffmpeg-skill/loudness, ffmpeg-skill/join. Which flags are used per tool is listed in skill --json →
    ffmpeg_skill.flags_used and checked against the live ffmpeg-skill contract by doctor.

Contract

audio-production skill --json prints audio-production/contract@1: stable identifiers skill_id, version,
tools[].tool_id, operations[].type, operations[].parameters, operations[].required_capabilities,
unsupported_operations, output_formats, errors.codes, errors.exit_codes, schema_versions. Everything is
derived from the tables the code runs on; there are no placeholder operations.

audio-production contract --check [FILE|-] verifies the printed contract against the implementation and the README
and, with a saved copy, classifies every difference as breaking (a pinned block, an operation's pinned fields, a
removed key) or additive (a new key outside the pinned blocks); exit 1 on problems or breaking drift. CI runs it
against tests/contract/contract.json; regenerate that file when a contract change is intended. contract/contract.json is the same output committed as
a file (python contract/generate.py), so a release's contract is git show vX.Y.Z:contract/contract.json. Pinned blocks and
the versioning rules: docs/contract.md.

provides lists this Skill's fifteen operations by their cross-repository Capability id (audio.gain,
audio.trim, audio.cut, audio.silence_remove, audio.fade_in, audio.fade_out, audio.normalize,
audio.mix, audio.mono, audio.stereo, audio.downmix, audio.noise_reduction, audio.dynamics,
audio.concat, audio.voice_cleanup), each with the operation it maps to, its tool_id (always audio-production/run, since this
Skill exposes one execution tool), and a lifecycle. This is a different vocabulary from the "Capabilities" above
(video-production-agent's environment/binary CapabilityResolver checks, e.g. ffmpeg, filter:<name>): provides
is for kajisho5/AI-video-production-OS's CapabilityContract.provides (see that project's docs/SPEC.md and
docs/decisions.md ADR-10 here), so a registry can resolve "who provides audio.gain" without hardcoding this
repository. It is additive and derived from model.OPERATION_TYPES; see ADR-10. Status: EXPERIMENTAL — the OS
specification it follows lives on that repository's main (docs/SPEC.md, docs/CAPABILITY_MATRIX.md); the ids
are kept in sync with docs/CAPABILITY_MATRIX.md there.

dependencies names the one Skill this one calls, in the OS registry's shape: ffmpeg-skill with a
version_range of its contract_version (>=1.0 <3.0: 1.0 up to ffmpeg-skill 2.x, 2.0 in 3.x; range_of: "contract_version") and the ffmpeg-skill tools
used. It is a declaration for the registry; compatibility is still decided flag by flag (ADR-15).

Input schema (audio-production/request@1)

{
  "schema": "audio-production/request@1",
  "project": {
    "project_id": "talk-42",
    "sources": [{"source_id": "mic", "path": "rec/mic.wav"}, {"source_id": "room", "path": "rec/room.wav"}],
    "tracks": [
      {"track_id": "voice", "source_id": "mic", "channel_layout": "mono"},
      {"track_id": "ambience", "source_id": "room", "range": {"start": 0.5, "end": 3600.0}, "gain_db": -12}
    ],
    "operations": [
      {"op_id": "trim", "type": "TRIM", "inputs": ["track:voice"], "parameters": {"start": 12.0, "end": 3612.0}},
      {"op_id": "cleanup", "type": "SILENCE_REMOVE", "inputs": ["op:trim"],
       "parameters": {"ranges": [{"start": 0.0, "end": 2.1}], "margin": 0.15, "min_duration": 0.6, "threshold_db": -40}},
      {"op_id": "nr", "type": "NOISE_REDUCTION", "inputs": ["op:cleanup"], "parameters": {"mode": "fft", "strength_db": 20}},
      {"op_id": "mix", "type": "MIX", "inputs": ["op:nr", "track:ambience"], "parameters": {"levels": [{"gain_db": 0}, {"gain_db": -6}]}},
      {"op_id": "fade", "type": "FADE_OUT", "inputs": ["op:mix"], "parameters": {"duration": 1.5}},
      {"op_id": "master", "type": "NORMALIZE", "inputs": ["op:fade"],
       "parameters": {"target_lufs": -16.0, "true_peak_db": -1.5, "tolerance_lufs": 1.0, "profile": "podcast"}}
    ],
    "outputs": [
      {"output_id": "master", "operation": "op:master", "path": "deliver/talk-42.wav", "format": "wav",
       "expect": {"channels": 1, "sample_rate": 48000}},
      {"output_id": "preview", "operation": "op:master", "path": "deliver/talk-42.m4a", "format": "m4a"}
    ]
  },
  "options": {"reuse_intermediates": true, "timeout": 900}
}
  • sources: files. tracks: generic tracks over a source (label, expected channel_layout, optional source
    range and gain_db, materialised as implicit TRIM / GAIN nodes). operations: the graph; inputs are references
    track:<id> / op:<id>. outputs: terminal artifacts with format and expectations.
  • Every id matches ^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$; times are seconds as numbers; ranges are half-open.
  • Fields named command, argv, args, cmd, shell, exec, executable, script, filter, filter_complex,
    af, env, cwd are rejected anywhere in the document; unknown fields are rejected everywhere.
  • Loudness: target_lufs and true_peak_db are required and have no default; the profile / caller decides.
    loudness_range_lu, tolerance_lufs (verification), sample_rate, profile (label, recorded only) are optional.
  • Silence: SILENCE_REMOVE executes the caller's ranges with margin / min_duration arithmetic and records
    threshold_db; it never detects silence and never decides that a range is unwanted.
  • Fillers: FILLER_REMOVE cuts at word timings the caller supplies (words, or a transcription-skill/transcript/0.1
    document as transcript) wherever a word matches the caller-chosen lexicon (presets ja / en, extra words,
    keep exceptions; required, no default). It never transcribes; a transcript segment without word timestamps is
    refused (INVALID_REQUEST, reason: word_timestamps_missing). See ADR-18. Words that are also ordinary words are
    in no preset; the contract publishes them as opt-in lists (filler_lexicons.opt_in, today only ja-ambiguous:
    あの / その / まあ / なんか) that a caller copies into words on purpose. Each removal of such a word is a warning and
    is listed in measurements.filler_remove.ambiguous. See ADR-25.

Output schema (audio-production/response@1)

{
  "schema": "audio-production/response@1", "skill": {"id": "audio-production", "version": "0.2.0"},
  "ok": true, "status": "ok", "dry_run": false,
  "plan": {"plan_id": "<sha256>", "graph": {"order": ["track:voice", "op:trim", "..."]}, "steps": ["..."], "required_capabilities": ["..."], "tool_versions": {"ffmpeg-skill": "0.12.2", "ffmpeg": "6.1.1"}},
  "results": [{
    "node_id": "op:master", "operation_id": "<sha256>", "type": "NORMALIZE", "tool": "ffmpeg-skill/loudness", "status": "completed",
    "parameters": {"target_lufs": -16.0, "true_peak_db": -1.5, "tolerance_lufs": 1.0, "profile": "podcast"},
    "inputs": ["op:fade"], "input_hashes": ["<sha256>"],
    "segments": [{"timeline": {"start": 0.0, "end": 3597.9}, "source_id": "mic", "source": {"start": 14.1, "end": 3612.0}, "input_index": 0}, "..."],
    "artifact": {"path": ".../.audio-production/talk-42/<id16>.wav", "duration": 3597.9, "channels": 1, "sample_rate": 48000, "codec": "pcm_s16le", "size": 0, "sha256": "<sha256>"},
    "measurements": {"loudness": {"measured_by": "ffmpeg-skill/loudness (NORMALIZE result)", "integrated_lufs": -16.0, "true_peak_dbtp": -1.6, "loudness_range_lu": 6.2}},
    "tool_commands_observed": ["ffmpeg ..."], "seconds": 12.3
  }],
  "outputs": [{"output_id": "master", "status": "completed", "path": ".../deliver/talk-42.wav", "format": "wav", "artifact": {"sha256": "<sha256>"}, "segments": ["..."],
               "provenance": {"skill": "audio-production", "skill_version": "0.2.0", "tool": "ffmpeg-skill/audio", "tool_versions": {}, "output_hash": "<sha256>",
                              "operation_id": "<sha256>", "operations": ["output -> operation -> ... -> source, with hashes and status"], "sources": {"mic": {"sha256": "<sha256>"}}}}],
  "tool_runs": [{"tool": "ffmpeg-skill/probe", "exit_code": 0, "seconds": 0.1, "commands_observed": []}],
  "warnings": []
}

Failure (any stage) is the same document shape with "ok": false, "status": "error" | "cancelled" and
"error": {"code", "message", "retryable", "details"}; the per-node results show which node failed and which
were skipped. ok mirrors the process exit code (0 ⇔ ok); status follows the media-analysis-skill convention.

Repository guide for future sessions: CLAUDE.md; current / experimental / planned state:
docs/STATE.md; contract versioning: docs/contract.md.

Supported operations

type inputs parameters ffmpeg-skill tool notes
GAIN 1 gain_db [−60, 60] audio --gain
TRIM 1 start, end (source time) cut --start/--end --accurate keeps [start, end); sample-accurate, measurements.cut.precision
CUT 1 remove: [{start,end}] sorted, non-overlapping cut --segments --accurate (kept ranges) sample-accurate
SILENCE_REMOVE 1 ranges, margin, min_duration, threshold_db (recorded) cut --segments --accurate ranges come from the caller
FILLER_REMOVE 1 words: [{start, end, text}] or transcript (one of), lexicon {presets?, words?, keep?} (required), pad [0, 0.5], merge_gap [0, 2], declick_ms [0, 200] cut --segments --accurate [--declick-ms] sample-accurate; matched words, ranges and removed seconds in measurements.filler_remove; declick_ms > 0 needs an ffmpeg-skill declaring --declick-ms (else TOOL_ERROR ffmpeg_skill_flag_missing at planning, ADR-18)
FADE_IN / FADE_OUT 1 duration audio --fade-in/--fade-out
NORMALIZE 1 target_lufs, true_peak_db (required), loudness_range_lu, tolerance_lufs, sample_rate, profile loudness -I/--tp/--lra/--sample-rate two-pass loudnorm, re-measured after
LOUDNESS_SET 1 versions: [{version_id, profile | target_lufs + true_peak_db, loudness_range_lu?, sample_rate?}] (1..8), tolerance_lufs (required) loudness -I/--tp/--lra/--sample-rate per version, loudness --measure-only per delivered output several delivery versions of one master in one run; a version is referenced as op:<op_id>.<version_id> (graph node type LOUDNESS_VERSION), the set itself is not an artifact. Profiles: ebu_r128 (−23 LUFS / −1 dBTP), atsc_a85 (−24 LKFS / −2 dBTP), regional arib_tr_b32 (−24 LKFS / −1 dBTP, Japan); none is a default, a streaming platform's value is a stated target (e.g. −14 / −1). Each version and each delivered file must measure within tolerance_lufs and at or under the ceiling (no slack), else VALIDATION_ERROR naming the version and nothing is delivered; only outputs may reference a version; response key loudness_sets (ADR-23)
MIX 2..8 levels: [{gain_db, mute}], duck {input, amount_db, threshold_db?, attack_ms?, release_ms?} audio --music/--music-volume (pairwise fold), --duck --duck-amount [--duck-threshold/--duck-attack/--duck-release] duration = first input (amix duration=first); duck lowers one bed under input 0 (sidechaincompress), other inputs are mixed un-ducked; undeclared duck flags → TOOL_ERROR ffmpeg_skill_flag_missing at planning (ADR-19)
STEMS 2..8 stems: [{stem_id, inputs: [input index], role?: dialogue|music|effects|other}] (1..8, every input in exactly one stem), levels: [{gain_db}] (as the MIX they sum to; no mute, no duck) audio --music/--music-volume (a stem with input 0), audio --replace [--music] (a stem without input 0: padded / trimmed to input 0's duration) one file per stem, referenced as op:<op_id>.<stem_id> (graph node type STEM); every stem has the programme's sample count (input 0's, as MIX), on the intermediate and the delivered file, else VALIDATION_ERROR stem_length_mismatch; stems are delivered as wav / flac only; a stem reaching 0 dBTP is refused (stem_clipped: a clipped stem does not sum to the mix); the stems summed equal a MIX of the same inputs and levels; response key stems. One multitrack WAV / MOV is not provided (no multitrack writer in ffmpeg-skill) (ADR-24)
MONO 1 – audio --mono requires a 2-channel input
STEREO 1 – audio --stereo requires a 1- or 2-channel input
DOWNMIX 1 – audio --downmix requires 5.1 / 7.1
NOISE_REDUCTION 1 mode: "fft", strength_db [20, 60] audio --denoise (afftdn) the only implemented mode; strength_db becomes afftdn nf=-strength_db, valid only in [−80, −20], so values below 20 are refused at validation (ADR-17)
DYNAMICS 1 gate {threshold_db, ratio, attack_ms, release_ms, range_db, knee_db}, compressor {threshold_db, ratio, attack_ms, release_ms, makeup_db, knee_db}, limiter {ceiling_db, attack_ms, release_ms} (≥ 1 stage) audio --gate/--compress/--limit (agate, acompressor, alimiter) fixed order gate → compressor → limiter; omitted fields keep ffmpeg's defaults
VOICE_CLEANUP 1 level: "light" | "medium" | "strong" (required, no default) audio --voice <level> (bare --voice = medium on ffmpeg-skill < 1.13) ffmpeg-skill's fixed speech chains: light = highpass 80 Hz + compressor; medium = highpass, de-esser, afftdn (fixed −25 dB floor), compressor; strong = medium + harder de-esser, second compressor, limiter at −1 dBFS. Nothing in a chain is tunable. A level the located ffmpeg-skill does not declare is refused at planning with TOOL_ERROR voice_level_unsupported (ADR-16)
CONCAT 2..32 crossfade (s, default 0), sample_rate, channels (1/2/6/8) join --transition none|fade --duration (concat / acrossfade) output layout = widest input unless channels; total = Σ − (n−1)·crossfade

Output formats (outputs[].format): wav (pcm_s16le), flac, mp3, m4a, aac, ogg, opus; encoder
availability is reported by doctor. Every intermediate is PCM WAV (no generation loss between operations).

Sources may be audio files or video containers (the audio track is extracted first; the output never carries video).

Declared but not implemented (unsupported_operations in the contract, UNSUPPORTED_OPERATION at validation):
CHANNEL_MAP (no typed mapping in ffmpeg-skill), RESAMPLE as a standalone operation (only NORMALIZE.sample_rate
and CONCAT.sample_rate), FORMAT_CONVERT (an output property). Details and the reasons:
docs/ffmpeg-skill.md.

CLI

command reads media writes media exit
skill --json / contract --json no no 0
contract --check [FILE|-] [--json] no no 0 ok / additive, 1 problems / breaking
doctor --json [--ffmpeg-skill DIR] [--workspace DIR] [--allowed-input ROOT] no (runs ffmpeg-skill doctor) no 0, 1 on fail
validate REQUEST|- --json no no 0 / error exit code
plan REQUEST|- --json probe only (read-only) no 0 / error exit code
run REQUEST|- --json [--dry-run] [--workspace DIR] [--allowed-input ROOT] [--ffmpeg-skill DIR] [--timeout S] [--no-reuse] [--cleanup keep|intermediates] yes yes 0 / error exit code

The ffmpeg-skill checkout is found from --ffmpeg-skill, else AUDIO_PRODUCTION_FFMPEG_SKILL_DIR,
VIDEO_AGENT_FFMPEG_SKILL_DIR, ~/.claude/skills/ffmpeg-skill, ./vendor/ffmpeg-skill, ../ffmpeg-skill. It is
never taken from the request document.

Doctor

doctor --json (audio-production/doctor@1) reports: Python; the located ffmpeg-skill (directory, version,
contract version, the flags this skill needs, problems); ffmpeg / ffprobe versions as detected by ffmpeg-skill's
doctor; every capability as supported / unsupported / unknown; every operation type with its status and the
missing or unknown capabilities; the not-implemented operations; output formats with encoder status; channel layout
and sample-rate policy; the path policy. Status: ok, degraded (some operation unsupported), fail (ffmpeg-skill
missing / incompatible or path policy broken). secrets_shown is always false.

Process boundary

caller → JSON (stdin) → typed validation → operation graph → ffmpeg-skill adapter (argv) → media execution → output validation → JSON (stdout). With --json, stdout carries exactly one document, on success and failure
alike; stderr carries diagnostics (including ffmpeg-skill's own log lines). Exit codes are stable (errors.exit_codes).

Security

  • No shell, no eval, no user-supplied command, argv, executable, script or filter string; the request schema
    rejects such fields by name and rejects unknown fields. One subprocess.Popen in the package (adapter), argv only,
    allow-listed scripts only, minimal child environment.
  • Every argv value is a number formatted by this skill (0.500, -6.000) or a resolved absolute path.
  • Inputs: regular files, symlinks resolved, optional --allowed-input roots; outputs: inside --workspace, no ..,
    no symlinked escape, no reserved Windows names / invalid characters / option-like names, never an input, never an
    existing file unless overwrite: true. Inputs are never modified.
  • Failed or cancelled runs leave no partial output and no intermediate without a manifest.
    Details: docs/security.md.

Determinism

Operation identity = sha256 over canonical JSON of {type, effective parameters, input identities, tool versions};
a source's identity is its file sha256. No timestamps, no UUIDs, stable topological order (smallest node id first
among ready nodes), stable key order. Identical request + identical inputs + identical ffmpeg-skill / ffmpeg version
→ identical plan_id and operation ids, reused intermediates, and content-equivalent outputs (bytes may differ
between encoder builds for lossy formats; WAV / FLAC are reproducible in practice).

Provenance

Per operation: operation_id, type, tool, tool_versions, parameters, input_hashes, output_hash,
segments (timeline ↔ source), status, measurements, tool_commands_observed. Per output: skill,
skill_version, tool, tool_versions, output_hash, the whole operation chain back to the sources with their
sha256. The caller's instruction (the request) and the skill's observation (results, measurements,
tool_runs) are separate parts of the response.

Error handling

code exit retryable when
INVALID_REQUEST 2 no document shape, unknown / forbidden field, bad type or range
INVALID_INPUT 3 no input missing, not a regular file, unreadable, no audio stream, video container
PATH_NOT_ALLOWED 4 no outside allowed roots / workspace, traversal, symlink escape, unsafe name
UNSUPPORTED_OPERATION 5 no not implemented (declared or unknown type, unsupported mode)
UNSUPPORTED_FORMAT 6 no output format unknown, extension mismatch, encoder missing
MISSING_INPUT 7 no reference to an undeclared source / track / operation
INVALID_TIME_RANGE 8 no end ≤ start, overlap, outside the media, nothing would remain, fade too long
INVALID_CHANNEL_LAYOUT 9 no unsupported layout, expectation mismatch, MONO / STEREO / DOWNMIX on the wrong input
INVALID_SAMPLE_RATE 10 no sample rate not in the accepted list
DEPENDENCY_ERROR 11 no duplicate id, cycle, self-reference, unreachable node
TOOL_ERROR 12 yes ffmpeg-skill missing / incompatible (not retryable), tool failure, timeout
OUTPUT_ERROR 13 no output exists, collides with an input, empty, could not be written
VALIDATION_ERROR 14 no artifact failed post-validation (stream, duration, channels, rate, codec, loudness)
CANCELLED 15 yes SIGINT / SIGTERM
INTERNAL_ERROR 16 no a bug in this skill (still one JSON document)

Testing

pip install -e . pytest
export AUDIO_PRODUCTION_FFMPEG_SKILL_DIR=/path/to/ffmpeg-skill     # or clone it as ../ffmpeg-skill
python -m pytest -q          # unit + security + contract + integration with real audio; nothing is skipped

tests/test_unit.py (schema, timeline arithmetic, graph, identities, canonical JSON, path policy, Windows names),
tests/test_security.py (no shell in source, script allow-list, argv formatting, injection through every parameter
kind, traversal / absolute / symlink escapes through the CLI, input-overwrite refusal, malformed JSON, minimal child
environment), tests/test_contract.py (contract ⇔ implementation, doctor honesty), tests/test_integration.py
(one positive and one negative case per operation, the pipeline below, a two-source mix, output validation, failure
handling, dry run, idempotent re-runs, timeout, SIGINT cancellation, CLI exit codes). CI runs Linux (3.9, 3.11),
Windows and macOS with a real FFmpeg and a fresh ffmpeg-skill clone: .github/workflows/tests.yml.

Real media verification

tests/test_integration.py::test_real_audio_pipeline_trim_gain_fade_normalize runs
source → TRIM → GAIN → FADE_OUT → NORMALIZE → output validation on a generated 6 s / 48 kHz PCM fixture through the
real ffmpeg-skill and FFmpeg and asserts duration, channels, sample rate, hashes and the re-measured loudness;
test_mix_two_sources_then_normalize runs A → GAIN, B (range, gain) → MIX → NORMALIZE; test_concat, test_dynamics
and the video-container cases cover the ffmpeg-skill capabilities this skill requires;
tests/test_ffmpeg_skill_compat.py pins the capability detection (any version, optional --overwrite /
--measure-only). Fixtures are synthesised with ffmpeg at test time (the tests may call ffmpeg; the skill never
does). Measured on ffmpeg 6.1.1 / ffmpeg-skill 0.9.1, re-verified against ffmpeg-skill 0.12.2 (commit 336e0c4) and
2.10.3; CI runs every cell against both 336e0c4 and ffmpeg-skill main.

Relationship to the other skills

  • ffmpeg-skill – the execution engine; see docs/ffmpeg-skill.md for the exact tools,
    flags and the compatibility gaps that define what is not offered here.
  • media-analysis-skill – measures; its silence and loudness observations are the natural inputs for
    SILENCE_REMOVE.ranges and NORMALIZE targets, chosen by the agent.
  • video-production-agent – the only intended caller: builds the request from its Decision / ProductionPlan,
    runs audio-production run - --json, reads results / outputs into its provenance and Artifact model.
  • transcription-skill – its transcript word timestamps are the intended input of FILLER_REMOVE (passed in the
    request; no code dependency, no speech recognition here).
  • video-editing-skill, QC – not touched; this skill has no video or final-QC role.

Current limitations

  • GAIN, FADE_IN, FADE_OUT, MONO, STEREO, DOWNMIX, NOISE_REDUCTION, DYNAMICS and VOICE_CLEANUP take an audio_stream
    parameter (0-based, default 0) selecting which audio stream of their one input to process (ffmpeg-skill/audio
    --audio-stream). TRIM, CUT, SILENCE_REMOVE, FILLER_REMOVE, NORMALIZE, MIX and CONCAT always use the first audio
    stream: ffmpeg-skill's cut / loudness / join tools have no --audio-stream flag, and MIX folds already-
    resolved inputs through --music, which always reads stream 0 of each bed file.
  • Every artifact's duration is validated within 0.1 s. Cuts are sample-accurate on PCM sources; on compressed
    sources the boundary depends on the decoder's priming handling (AAC: exact on ffmpeg 6.1, −12 ms on the Windows
    CI runner's FFmpeg), and CONCAT / MIX / export may differ by a codec frame (measured +8–10 ms with AAC sources).
  • Intermediates are 16-bit PCM WAV. MIX output duration follows the first input; per-input pan is not available;
    duck lowers exactly one bed, keyed by input 0 (each fold is one ffmpeg-skill call keyed by its own main track).
  • FILLER_REMOVE word times are on its input's timeline (a transcript of the source fits an input that has not been
    trimmed or cut before); matching is exact on normalised text, so a filler the transcript spells differently from
    the lexicon is kept.
  • Loudness true-peak of a lossy output (m4a, mp3, …) may exceed the ceiling by the codec's overshoot; the
    verification happens on the PCM intermediate.
  • doctor reports core ffmpeg filters (volume, afade, amix, pan, aformat) as unknown because
    ffmpeg-skill's doctor does not probe them; they are verified by output validation at run time. On FFmpeg ≥ 8.0
    ffmpeg-skill 0.9's doctor parses no filters at all (-filters prints two flag characters, its pattern expects
    three), so every filter capability is unknown there (checks.filter_detection), never falsely unsupported.
  • Intermediates are kept by default so identical re-runs reuse them; run --cleanup intermediates removes the
    project's work directory after a fully successful run (never on failure, never outputs; reported under cleanup
    in the response). There is no age- or size-based eviction across projects.

Future extensions (not in this release)

Typed channel mapping, standalone resampling, more noise-reduction modes with detected capabilities, per-input pan
in MIX, 24-bit intermediates — each requires a corresponding capability in ffmpeg-skill's public contract (or a
decision to add a second execution backend), and will be declared only once implemented and tested. (Audio-stream
selection is no longer on this list: ffmpeg-skill/audio --audio-stream has existed since 0.9.1, and audio_stream
is implemented above on every operation that reads its input through audio.py.)

Repository guide for future sessions: CLAUDE.md; current / experimental / planned state:
docs/STATE.md; contract versioning: docs/contract.md.

Support

If this skill saves you time, you can help keep it maintained through GitHub Sponsors.
Issues and pull requests are just as welcome.

License

MIT

Reviews (0)

No results found