audio-production-skill
Health Warn
- License — License: MIT
- Description — Repository has a description
- Active repo — Last push 0 days ago
- Low visibility — Only 6 GitHub stars
Code Pass
- Code scan — Scanned 12 files during light audit, no dangerous patterns found
Permissions Pass
- Permissions — No dangerous permissions requested
No AI report is available for this listing yet.
Agent skill that deterministically executes typed audio operation graphs via ffmpeg-skill.
Deterministic audio production / processing Skill for the AI Video Production Ecosystem
Executes a typed operation graph through ffmpeg-skill, with validated artifacts and provenance out.
NOT an AI agent: no LLM, no prompt, no reasoning, no decision.
CLI · Supported operations · Architecture · Contract
Deterministic audio production / processing Skill for the AI Video Production Ecosystem: gain, sample-accurate
trim / cut, silence removal (explicit ranges), fades, EBU R128 loudness normalisation, mix, concat, mono / stereo /
surround down-mix, FFT noise reduction, typed dynamics (gate / compressor / limiter), format conversion and audio
extraction from video containers, executed as a typed operation graph through
ffmpeg-skill, with validated artifacts and provenance out.
audio-production-skill is NOT an AI agent. It contains no LLM, no prompt, no reasoning, no decision, no
production plan. It executes what a typed request says, refuses everything else, and reports what it observed.
audio-production skill --json # Skill / Capability / Tool contract (alias: contract --json)
audio-production doctor --json # environment vs. contract: ffmpeg-skill, ffmpeg, capabilities per operation
audio-production validate - --json # validate a request document, run nothing, read no media
audio-production plan - --json # dry run: graph, tool selection, expected timeline; writes no media
audio-production run - --json # execute (stdin: request document; stdout: exactly one response document)
Requirements: Python 3.9+, standard library only; an ffmpeg-skill checkout (any version: there is no version
window — compatibility is detected from the checkout's own contract, which must declare probe, audio, cut,loudness and join with every flag this skill emits; optional flags such as --overwrite are used only where
declared, see docs/ffmpeg-skill.md; CI runs against both commit 336e0c4 (0.12.2) and ffmpeg-skill main) and
FFmpeg (ffmpeg + ffprobe) on PATH for ffmpeg-skill. Install: pip install -e .
What it is, and what it is not
| ffmpeg-skill | media-analysis-skill | audio-production-skill | video-production-agent | |
|---|---|---|---|---|
| Role | low-level media execution engine (hands) | measurement / observation (meters) | audio production execution (typed audio operations) | reasoning / decision / planning / orchestration (brain) |
| Does | runs ffmpeg for cut, audio post, loudness, join, export, … | measures loudness, silence, streams, integrity | executes GAIN / TRIM / CUT / SILENCE_REMOVE / FILLER_REMOVE / FADE / NORMALIZE / LOUDNESS_SET / MIX (with ducking) / STEMS / CONCAT / MONO / STEREO / DOWNMIX / NOISE_REDUCTION / DYNAMICS / VOICE_CLEANUP as a dependency graph, validates every artifact, records provenance | decides whether and what to process, builds the request |
| Never | holds a project model | edits or writes media | measures for decisions, decides which ranges are silence, recognises speech, invents parameters, runs ffmpeg directly, accepts commands / filters | runs ffmpeg |
- audio-production-skill ≠ media-analysis-skill. media-analysis-skill says "silence from 0.0 to 2.0 s, −23.1 LUFS".
audio-production-skill only executes "remove [0.0, 2.0)", "normalise to −16 LUFS / −1.5 dBTP". The one measurement
this skill performs is the verification of its own NORMALIZE and LOUDNESS_SET output (re-measured through ffmpeg-skill/loudness);
it never reports a measurement of an input for a decision. - audio-production-skill ≠ ffmpeg-skill. ffmpeg-skill runs FFmpeg per script call. audio-production-skill owns the
audio project model: sources, tracks, operation graph with deterministic identities, source ↔ timeline mapping,
intermediate management, output validation and provenance. It never callsffmpegitself: every process it starts
ispython3 <ffmpeg-skill>/scripts/{probe,audio,cut,loudness,join}.pywith a typed argv. - Loudness: owner and engine. This repository owns 音量・ラウドネス; ffmpeg-skill is the engine.
NORMALIZEandLOUDNESS_SETrun through ffmpeg-skillloudness. The
agent's video path keeps callingffmpeg-skill/loudnessdirectly; that routing changes only after a measured comparison
ofloudnormandebur128on the same files (video-production-agent step 34). That comparison is committed
(ADR-116, #160): on the same fixtures both paths write the same samples, on target by both meters, so the
routing stays as it is. See the
Velisa ownership decisions 2026-11-12 (D23). - audio-production-skill ≠ video-production-agent. There is no Observation → Inference → Decision → ProductionPlan
here. The agent decides; this skill executes one typed request and says exactly what happened.
Architecture
external caller (video-production-agent adapter)
│ JSON request audio-production/request@1 (stdin)
▼
audio-production run - --json
├─ model.parse_request typed validation: schema, ids, references, parameter schemas per operation type,
│ forbidden fields (command / argv / filter / …), formats, layouts, sample rates
├─ security.PathPolicy inputs are regular files (symlinks resolved, optional allowed roots);
│ outputs inside the workspace, never an input, never an existing file unless overwrite
├─ graph.OperationGraph nodes (tracks, implicit track range / gain, operations), deterministic topological
│ order, cycle / unreachable detection
├─ adapter.FfmpegSkill.probe every source probed (read-only) and sha256-fingerprinted; a video container's audio
│ track is extracted to a PCM WAV intermediate (ffmpeg-skill/audio)
├─ graph.identities operation_id = sha256(type, parameters, input identities, tool versions)
├─ plan tool per node, argv template, expected timeline (segments) and duration
├─ execute (in order) ffmpeg-skill/{audio,cut,loudness,join} → PCM WAV intermediate, reused when the identity
│ and input hashes match a manifest; sidecars and partial files removed on failure
├─ validate exists, size > 0, readable, audio stream, codec, duration ± tolerance, channels,
│ sample rate, sha256; NORMALIZE re-measured (loudness / true peak vs. tolerance)
├─ export outputs ffmpeg-skill/audio to the requested format, validated against `expect`
▼
JSON response audio-production/response@1 (stdout, exactly one document; stderr = diagnostics)
Full description: docs/architecture.md.
Skill / Capability / Tool
- Skill:
audio-production(this package), kindexecution, one toolaudio-production/run. - Capabilities (vocabulary of video-production-agent's CapabilityResolver):
ffmpeg-skill,ffmpeg,ffprobe,filter:<name>,encoder:<name>.doctorreports each assupported,unsupportedorunknown(core ffmpeg
filters are not probed by ffmpeg-skill's doctor and are therefore reportedunknown, then verified per run). - Tools used (all through ffmpeg-skill's public contract 1.0):
ffmpeg-skill/probe,ffmpeg-skill/audio,ffmpeg-skill/cut,ffmpeg-skill/loudness,ffmpeg-skill/join. Which flags are used per tool is listed inskill --json→ffmpeg_skill.flags_usedand checked against the live ffmpeg-skill contract bydoctor.
Contract
audio-production skill --json prints audio-production/contract@1: stable identifiers skill_id, version,tools[].tool_id, operations[].type, operations[].parameters, operations[].required_capabilities,unsupported_operations, output_formats, errors.codes, errors.exit_codes, schema_versions. Everything is
derived from the tables the code runs on; there are no placeholder operations.
audio-production contract --check [FILE|-] verifies the printed contract against the implementation and the README
and, with a saved copy, classifies every difference as breaking (a pinned block, an operation's pinned fields, a
removed key) or additive (a new key outside the pinned blocks); exit 1 on problems or breaking drift. CI runs it
against tests/contract/contract.json; regenerate that file when a contract change is intended. contract/contract.json is the same output committed as
a file (python contract/generate.py), so a release's contract is git show vX.Y.Z:contract/contract.json. Pinned blocks and
the versioning rules: docs/contract.md.
provides lists this Skill's fifteen operations by their cross-repository Capability id (audio.gain,audio.trim, audio.cut, audio.silence_remove, audio.fade_in, audio.fade_out, audio.normalize,audio.mix, audio.mono, audio.stereo, audio.downmix, audio.noise_reduction, audio.dynamics,audio.concat, audio.voice_cleanup), each with the operation it maps to, its tool_id (always audio-production/run, since this
Skill exposes one execution tool), and a lifecycle. This is a different vocabulary from the "Capabilities" above
(video-production-agent's environment/binary CapabilityResolver checks, e.g. ffmpeg, filter:<name>): provides
is for kajisho5/AI-video-production-OS's CapabilityContract.provides (see that project's docs/SPEC.md anddocs/decisions.md ADR-10 here), so a registry can resolve "who provides audio.gain" without hardcoding this
repository. It is additive and derived from model.OPERATION_TYPES; see ADR-10. Status: EXPERIMENTAL — the OS
specification it follows lives on that repository's main (docs/SPEC.md, docs/CAPABILITY_MATRIX.md); the ids
are kept in sync with docs/CAPABILITY_MATRIX.md there.
dependencies names the one Skill this one calls, in the OS registry's shape: ffmpeg-skill with aversion_range of its contract_version (>=1.0 <3.0: 1.0 up to ffmpeg-skill 2.x, 2.0 in 3.x; range_of: "contract_version") and the ffmpeg-skill tools
used. It is a declaration for the registry; compatibility is still decided flag by flag (ADR-15).
Input schema (audio-production/request@1)
{
"schema": "audio-production/request@1",
"project": {
"project_id": "talk-42",
"sources": [{"source_id": "mic", "path": "rec/mic.wav"}, {"source_id": "room", "path": "rec/room.wav"}],
"tracks": [
{"track_id": "voice", "source_id": "mic", "channel_layout": "mono"},
{"track_id": "ambience", "source_id": "room", "range": {"start": 0.5, "end": 3600.0}, "gain_db": -12}
],
"operations": [
{"op_id": "trim", "type": "TRIM", "inputs": ["track:voice"], "parameters": {"start": 12.0, "end": 3612.0}},
{"op_id": "cleanup", "type": "SILENCE_REMOVE", "inputs": ["op:trim"],
"parameters": {"ranges": [{"start": 0.0, "end": 2.1}], "margin": 0.15, "min_duration": 0.6, "threshold_db": -40}},
{"op_id": "nr", "type": "NOISE_REDUCTION", "inputs": ["op:cleanup"], "parameters": {"mode": "fft", "strength_db": 20}},
{"op_id": "mix", "type": "MIX", "inputs": ["op:nr", "track:ambience"], "parameters": {"levels": [{"gain_db": 0}, {"gain_db": -6}]}},
{"op_id": "fade", "type": "FADE_OUT", "inputs": ["op:mix"], "parameters": {"duration": 1.5}},
{"op_id": "master", "type": "NORMALIZE", "inputs": ["op:fade"],
"parameters": {"target_lufs": -16.0, "true_peak_db": -1.5, "tolerance_lufs": 1.0, "profile": "podcast"}}
],
"outputs": [
{"output_id": "master", "operation": "op:master", "path": "deliver/talk-42.wav", "format": "wav",
"expect": {"channels": 1, "sample_rate": 48000}},
{"output_id": "preview", "operation": "op:master", "path": "deliver/talk-42.m4a", "format": "m4a"}
]
},
"options": {"reuse_intermediates": true, "timeout": 900}
}
sources: files.tracks: generic tracks over a source (label, expectedchannel_layout, optional sourcerangeandgain_db, materialised as implicit TRIM / GAIN nodes).operations: the graph; inputs are referencestrack:<id>/op:<id>.outputs: terminal artifacts with format and expectations.- Every id matches
^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$; times are seconds as numbers; ranges are half-open. - Fields named
command,argv,args,cmd,shell,exec,executable,script,filter,filter_complex,af,env,cwdare rejected anywhere in the document; unknown fields are rejected everywhere. - Loudness:
target_lufsandtrue_peak_dbare required and have no default; the profile / caller decides.loudness_range_lu,tolerance_lufs(verification),sample_rate,profile(label, recorded only) are optional. - Silence:
SILENCE_REMOVEexecutes the caller's ranges withmargin/min_durationarithmetic and recordsthreshold_db; it never detects silence and never decides that a range is unwanted. - Fillers:
FILLER_REMOVEcuts at word timings the caller supplies (words, or atranscription-skill/transcript/0.1
document astranscript) wherever a word matches the caller-chosenlexicon(presetsja/en, extrawords,keepexceptions; required, no default). It never transcribes; a transcript segment without word timestamps is
refused (INVALID_REQUEST,reason: word_timestamps_missing). See ADR-18. Words that are also ordinary words are
in no preset; the contract publishes them as opt-in lists (filler_lexicons.opt_in, today onlyja-ambiguous:
あの / その / まあ / なんか) that a caller copies intowordson purpose. Each removal of such a word is a warning and
is listed inmeasurements.filler_remove.ambiguous. See ADR-25.
Output schema (audio-production/response@1)
{
"schema": "audio-production/response@1", "skill": {"id": "audio-production", "version": "0.2.0"},
"ok": true, "status": "ok", "dry_run": false,
"plan": {"plan_id": "<sha256>", "graph": {"order": ["track:voice", "op:trim", "..."]}, "steps": ["..."], "required_capabilities": ["..."], "tool_versions": {"ffmpeg-skill": "0.12.2", "ffmpeg": "6.1.1"}},
"results": [{
"node_id": "op:master", "operation_id": "<sha256>", "type": "NORMALIZE", "tool": "ffmpeg-skill/loudness", "status": "completed",
"parameters": {"target_lufs": -16.0, "true_peak_db": -1.5, "tolerance_lufs": 1.0, "profile": "podcast"},
"inputs": ["op:fade"], "input_hashes": ["<sha256>"],
"segments": [{"timeline": {"start": 0.0, "end": 3597.9}, "source_id": "mic", "source": {"start": 14.1, "end": 3612.0}, "input_index": 0}, "..."],
"artifact": {"path": ".../.audio-production/talk-42/<id16>.wav", "duration": 3597.9, "channels": 1, "sample_rate": 48000, "codec": "pcm_s16le", "size": 0, "sha256": "<sha256>"},
"measurements": {"loudness": {"measured_by": "ffmpeg-skill/loudness (NORMALIZE result)", "integrated_lufs": -16.0, "true_peak_dbtp": -1.6, "loudness_range_lu": 6.2}},
"tool_commands_observed": ["ffmpeg ..."], "seconds": 12.3
}],
"outputs": [{"output_id": "master", "status": "completed", "path": ".../deliver/talk-42.wav", "format": "wav", "artifact": {"sha256": "<sha256>"}, "segments": ["..."],
"provenance": {"skill": "audio-production", "skill_version": "0.2.0", "tool": "ffmpeg-skill/audio", "tool_versions": {}, "output_hash": "<sha256>",
"operation_id": "<sha256>", "operations": ["output -> operation -> ... -> source, with hashes and status"], "sources": {"mic": {"sha256": "<sha256>"}}}}],
"tool_runs": [{"tool": "ffmpeg-skill/probe", "exit_code": 0, "seconds": 0.1, "commands_observed": []}],
"warnings": []
}
Failure (any stage) is the same document shape with "ok": false, "status": "error" | "cancelled" and"error": {"code", "message", "retryable", "details"}; the per-node results show which node failed and which
were skipped. ok mirrors the process exit code (0 ⇔ ok); status follows the media-analysis-skill convention.
Repository guide for future sessions: CLAUDE.md; current / experimental / planned state:
docs/STATE.md; contract versioning: docs/contract.md.
Supported operations
| type | inputs | parameters | ffmpeg-skill tool | notes |
|---|---|---|---|---|
GAIN |
1 | gain_db [−60, 60] |
audio --gain |
|
TRIM |
1 | start, end (source time) |
cut --start/--end --accurate |
keeps [start, end); sample-accurate, measurements.cut.precision |
CUT |
1 | remove: [{start,end}] sorted, non-overlapping |
cut --segments --accurate (kept ranges) |
sample-accurate |
SILENCE_REMOVE |
1 | ranges, margin, min_duration, threshold_db (recorded) |
cut --segments --accurate |
ranges come from the caller |
FILLER_REMOVE |
1 | words: [{start, end, text}] or transcript (one of), lexicon {presets?, words?, keep?} (required), pad [0, 0.5], merge_gap [0, 2], declick_ms [0, 200] |
cut --segments --accurate [--declick-ms] |
sample-accurate; matched words, ranges and removed seconds in measurements.filler_remove; declick_ms > 0 needs an ffmpeg-skill declaring --declick-ms (else TOOL_ERROR ffmpeg_skill_flag_missing at planning, ADR-18) |
FADE_IN / FADE_OUT |
1 | duration |
audio --fade-in/--fade-out |
|
NORMALIZE |
1 | target_lufs, true_peak_db (required), loudness_range_lu, tolerance_lufs, sample_rate, profile |
loudness -I/--tp/--lra/--sample-rate |
two-pass loudnorm, re-measured after |
LOUDNESS_SET |
1 | versions: [{version_id, profile | target_lufs + true_peak_db, loudness_range_lu?, sample_rate?}] (1..8), tolerance_lufs (required) |
loudness -I/--tp/--lra/--sample-rate per version, loudness --measure-only per delivered output |
several delivery versions of one master in one run; a version is referenced as op:<op_id>.<version_id> (graph node type LOUDNESS_VERSION), the set itself is not an artifact. Profiles: ebu_r128 (−23 LUFS / −1 dBTP), atsc_a85 (−24 LKFS / −2 dBTP), regional arib_tr_b32 (−24 LKFS / −1 dBTP, Japan); none is a default, a streaming platform's value is a stated target (e.g. −14 / −1). Each version and each delivered file must measure within tolerance_lufs and at or under the ceiling (no slack), else VALIDATION_ERROR naming the version and nothing is delivered; only outputs may reference a version; response key loudness_sets (ADR-23) |
MIX |
2..8 | levels: [{gain_db, mute}], duck {input, amount_db, threshold_db?, attack_ms?, release_ms?} |
audio --music/--music-volume (pairwise fold), --duck --duck-amount [--duck-threshold/--duck-attack/--duck-release] |
duration = first input (amix duration=first); duck lowers one bed under input 0 (sidechaincompress), other inputs are mixed un-ducked; undeclared duck flags → TOOL_ERROR ffmpeg_skill_flag_missing at planning (ADR-19) |
STEMS |
2..8 | stems: [{stem_id, inputs: [input index], role?: dialogue|music|effects|other}] (1..8, every input in exactly one stem), levels: [{gain_db}] (as the MIX they sum to; no mute, no duck) |
audio --music/--music-volume (a stem with input 0), audio --replace [--music] (a stem without input 0: padded / trimmed to input 0's duration) |
one file per stem, referenced as op:<op_id>.<stem_id> (graph node type STEM); every stem has the programme's sample count (input 0's, as MIX), on the intermediate and the delivered file, else VALIDATION_ERROR stem_length_mismatch; stems are delivered as wav / flac only; a stem reaching 0 dBTP is refused (stem_clipped: a clipped stem does not sum to the mix); the stems summed equal a MIX of the same inputs and levels; response key stems. One multitrack WAV / MOV is not provided (no multitrack writer in ffmpeg-skill) (ADR-24) |
MONO |
1 | – | audio --mono |
requires a 2-channel input |
STEREO |
1 | – | audio --stereo |
requires a 1- or 2-channel input |
DOWNMIX |
1 | – | audio --downmix |
requires 5.1 / 7.1 |
NOISE_REDUCTION |
1 | mode: "fft", strength_db [20, 60] |
audio --denoise (afftdn) |
the only implemented mode; strength_db becomes afftdn nf=-strength_db, valid only in [−80, −20], so values below 20 are refused at validation (ADR-17) |
DYNAMICS |
1 | gate {threshold_db, ratio, attack_ms, release_ms, range_db, knee_db}, compressor {threshold_db, ratio, attack_ms, release_ms, makeup_db, knee_db}, limiter {ceiling_db, attack_ms, release_ms} (≥ 1 stage) |
audio --gate/--compress/--limit (agate, acompressor, alimiter) |
fixed order gate → compressor → limiter; omitted fields keep ffmpeg's defaults |
VOICE_CLEANUP |
1 | level: "light" | "medium" | "strong" (required, no default) |
audio --voice <level> (bare --voice = medium on ffmpeg-skill < 1.13) |
ffmpeg-skill's fixed speech chains: light = highpass 80 Hz + compressor; medium = highpass, de-esser, afftdn (fixed −25 dB floor), compressor; strong = medium + harder de-esser, second compressor, limiter at −1 dBFS. Nothing in a chain is tunable. A level the located ffmpeg-skill does not declare is refused at planning with TOOL_ERROR voice_level_unsupported (ADR-16) |
CONCAT |
2..32 | crossfade (s, default 0), sample_rate, channels (1/2/6/8) |
join --transition none|fade --duration (concat / acrossfade) |
output layout = widest input unless channels; total = Σ − (n−1)·crossfade |
Output formats (outputs[].format): wav (pcm_s16le), flac, mp3, m4a, aac, ogg, opus; encoder
availability is reported by doctor. Every intermediate is PCM WAV (no generation loss between operations).
Sources may be audio files or video containers (the audio track is extracted first; the output never carries video).
Declared but not implemented (unsupported_operations in the contract, UNSUPPORTED_OPERATION at validation):CHANNEL_MAP (no typed mapping in ffmpeg-skill), RESAMPLE as a standalone operation (only NORMALIZE.sample_rate
and CONCAT.sample_rate), FORMAT_CONVERT (an output property). Details and the reasons:
docs/ffmpeg-skill.md.
CLI
| command | reads media | writes media | exit |
|---|---|---|---|
skill --json / contract --json |
no | no | 0 |
contract --check [FILE|-] [--json] |
no | no | 0 ok / additive, 1 problems / breaking |
doctor --json [--ffmpeg-skill DIR] [--workspace DIR] [--allowed-input ROOT] |
no (runs ffmpeg-skill doctor) | no | 0, 1 on fail |
validate REQUEST|- --json |
no | no | 0 / error exit code |
plan REQUEST|- --json |
probe only (read-only) | no | 0 / error exit code |
run REQUEST|- --json [--dry-run] [--workspace DIR] [--allowed-input ROOT] [--ffmpeg-skill DIR] [--timeout S] [--no-reuse] [--cleanup keep|intermediates] |
yes | yes | 0 / error exit code |
The ffmpeg-skill checkout is found from --ffmpeg-skill, else AUDIO_PRODUCTION_FFMPEG_SKILL_DIR,VIDEO_AGENT_FFMPEG_SKILL_DIR, ~/.claude/skills/ffmpeg-skill, ./vendor/ffmpeg-skill, ../ffmpeg-skill. It is
never taken from the request document.
Doctor
doctor --json (audio-production/doctor@1) reports: Python; the located ffmpeg-skill (directory, version,
contract version, the flags this skill needs, problems); ffmpeg / ffprobe versions as detected by ffmpeg-skill's
doctor; every capability as supported / unsupported / unknown; every operation type with its status and the
missing or unknown capabilities; the not-implemented operations; output formats with encoder status; channel layout
and sample-rate policy; the path policy. Status: ok, degraded (some operation unsupported), fail (ffmpeg-skill
missing / incompatible or path policy broken). secrets_shown is always false.
Process boundary
caller → JSON (stdin) → typed validation → operation graph → ffmpeg-skill adapter (argv) → media execution → output validation → JSON (stdout). With --json, stdout carries exactly one document, on success and failure
alike; stderr carries diagnostics (including ffmpeg-skill's own log lines). Exit codes are stable (errors.exit_codes).
Security
- No shell, no
eval, no user-supplied command, argv, executable, script or filter string; the request schema
rejects such fields by name and rejects unknown fields. Onesubprocess.Popenin the package (adapter), argv only,
allow-listed scripts only, minimal child environment. - Every argv value is a number formatted by this skill (
0.500,-6.000) or a resolved absolute path. - Inputs: regular files, symlinks resolved, optional
--allowed-inputroots; outputs: inside--workspace, no..,
no symlinked escape, no reserved Windows names / invalid characters / option-like names, never an input, never an
existing file unlessoverwrite: true. Inputs are never modified. - Failed or cancelled runs leave no partial output and no intermediate without a manifest.
Details: docs/security.md.
Determinism
Operation identity = sha256 over canonical JSON of {type, effective parameters, input identities, tool versions};
a source's identity is its file sha256. No timestamps, no UUIDs, stable topological order (smallest node id first
among ready nodes), stable key order. Identical request + identical inputs + identical ffmpeg-skill / ffmpeg version
→ identical plan_id and operation ids, reused intermediates, and content-equivalent outputs (bytes may differ
between encoder builds for lossy formats; WAV / FLAC are reproducible in practice).
Provenance
Per operation: operation_id, type, tool, tool_versions, parameters, input_hashes, output_hash,segments (timeline ↔ source), status, measurements, tool_commands_observed. Per output: skill,skill_version, tool, tool_versions, output_hash, the whole operation chain back to the sources with their
sha256. The caller's instruction (the request) and the skill's observation (results, measurements,tool_runs) are separate parts of the response.
Error handling
| code | exit | retryable | when |
|---|---|---|---|
INVALID_REQUEST |
2 | no | document shape, unknown / forbidden field, bad type or range |
INVALID_INPUT |
3 | no | input missing, not a regular file, unreadable, no audio stream, video container |
PATH_NOT_ALLOWED |
4 | no | outside allowed roots / workspace, traversal, symlink escape, unsafe name |
UNSUPPORTED_OPERATION |
5 | no | not implemented (declared or unknown type, unsupported mode) |
UNSUPPORTED_FORMAT |
6 | no | output format unknown, extension mismatch, encoder missing |
MISSING_INPUT |
7 | no | reference to an undeclared source / track / operation |
INVALID_TIME_RANGE |
8 | no | end ≤ start, overlap, outside the media, nothing would remain, fade too long |
INVALID_CHANNEL_LAYOUT |
9 | no | unsupported layout, expectation mismatch, MONO / STEREO / DOWNMIX on the wrong input |
INVALID_SAMPLE_RATE |
10 | no | sample rate not in the accepted list |
DEPENDENCY_ERROR |
11 | no | duplicate id, cycle, self-reference, unreachable node |
TOOL_ERROR |
12 | yes | ffmpeg-skill missing / incompatible (not retryable), tool failure, timeout |
OUTPUT_ERROR |
13 | no | output exists, collides with an input, empty, could not be written |
VALIDATION_ERROR |
14 | no | artifact failed post-validation (stream, duration, channels, rate, codec, loudness) |
CANCELLED |
15 | yes | SIGINT / SIGTERM |
INTERNAL_ERROR |
16 | no | a bug in this skill (still one JSON document) |
Testing
pip install -e . pytest
export AUDIO_PRODUCTION_FFMPEG_SKILL_DIR=/path/to/ffmpeg-skill # or clone it as ../ffmpeg-skill
python -m pytest -q # unit + security + contract + integration with real audio; nothing is skipped
tests/test_unit.py (schema, timeline arithmetic, graph, identities, canonical JSON, path policy, Windows names),tests/test_security.py (no shell in source, script allow-list, argv formatting, injection through every parameter
kind, traversal / absolute / symlink escapes through the CLI, input-overwrite refusal, malformed JSON, minimal child
environment), tests/test_contract.py (contract ⇔ implementation, doctor honesty), tests/test_integration.py
(one positive and one negative case per operation, the pipeline below, a two-source mix, output validation, failure
handling, dry run, idempotent re-runs, timeout, SIGINT cancellation, CLI exit codes). CI runs Linux (3.9, 3.11),
Windows and macOS with a real FFmpeg and a fresh ffmpeg-skill clone: .github/workflows/tests.yml.
Real media verification
tests/test_integration.py::test_real_audio_pipeline_trim_gain_fade_normalize runssource → TRIM → GAIN → FADE_OUT → NORMALIZE → output validation on a generated 6 s / 48 kHz PCM fixture through the
real ffmpeg-skill and FFmpeg and asserts duration, channels, sample rate, hashes and the re-measured loudness;test_mix_two_sources_then_normalize runs A → GAIN, B (range, gain) → MIX → NORMALIZE; test_concat, test_dynamics
and the video-container cases cover the ffmpeg-skill capabilities this skill requires;tests/test_ffmpeg_skill_compat.py pins the capability detection (any version, optional --overwrite /--measure-only). Fixtures are synthesised with ffmpeg at test time (the tests may call ffmpeg; the skill never
does). Measured on ffmpeg 6.1.1 / ffmpeg-skill 0.9.1, re-verified against ffmpeg-skill 0.12.2 (commit 336e0c4) and
2.10.3; CI runs every cell against both 336e0c4 and ffmpeg-skill main.
Relationship to the other skills
- ffmpeg-skill – the execution engine; see docs/ffmpeg-skill.md for the exact tools,
flags and the compatibility gaps that define what is not offered here. - media-analysis-skill – measures; its
silenceandloudnessobservations are the natural inputs forSILENCE_REMOVE.rangesandNORMALIZEtargets, chosen by the agent. - video-production-agent – the only intended caller: builds the request from its Decision / ProductionPlan,
runsaudio-production run - --json, readsresults/outputsinto its provenance and Artifact model. - transcription-skill – its transcript word timestamps are the intended input of
FILLER_REMOVE(passed in the
request; no code dependency, no speech recognition here). - video-editing-skill, QC – not touched; this skill has no video or final-QC role.
Current limitations
GAIN,FADE_IN,FADE_OUT,MONO,STEREO,DOWNMIX,NOISE_REDUCTION,DYNAMICSandVOICE_CLEANUPtake anaudio_stream
parameter (0-based, default 0) selecting which audio stream of their one input to process (ffmpeg-skill/audio--audio-stream).TRIM,CUT,SILENCE_REMOVE,FILLER_REMOVE,NORMALIZE,MIXandCONCATalways use the first audio
stream: ffmpeg-skill'scut/loudness/jointools have no--audio-streamflag, andMIXfolds already-
resolved inputs through--music, which always reads stream 0 of each bed file.- Every artifact's duration is validated within 0.1 s. Cuts are sample-accurate on PCM sources; on compressed
sources the boundary depends on the decoder's priming handling (AAC: exact on ffmpeg 6.1, −12 ms on the Windows
CI runner's FFmpeg), and CONCAT / MIX / export may differ by a codec frame (measured +8–10 ms with AAC sources). - Intermediates are 16-bit PCM WAV. MIX output duration follows the first input; per-input pan is not available;
ducklowers exactly one bed, keyed by input 0 (each fold is one ffmpeg-skill call keyed by its own main track). FILLER_REMOVEword times are on its input's timeline (a transcript of the source fits an input that has not been
trimmed or cut before); matching is exact on normalised text, so a filler the transcript spells differently from
the lexicon is kept.- Loudness true-peak of a lossy output (
m4a,mp3, …) may exceed the ceiling by the codec's overshoot; the
verification happens on the PCM intermediate. doctorreports core ffmpeg filters (volume,afade,amix,pan,aformat) asunknownbecause
ffmpeg-skill's doctor does not probe them; they are verified by output validation at run time. On FFmpeg ≥ 8.0
ffmpeg-skill 0.9's doctor parses no filters at all (-filtersprints two flag characters, its pattern expects
three), so every filter capability isunknownthere (checks.filter_detection), never falselyunsupported.- Intermediates are kept by default so identical re-runs reuse them;
run --cleanup intermediatesremoves the
project's work directory after a fully successful run (never on failure, never outputs; reported undercleanup
in the response). There is no age- or size-based eviction across projects.
Future extensions (not in this release)
Typed channel mapping, standalone resampling, more noise-reduction modes with detected capabilities, per-input pan
in MIX, 24-bit intermediates — each requires a corresponding capability in ffmpeg-skill's public contract (or a
decision to add a second execution backend), and will be declared only once implemented and tested. (Audio-stream
selection is no longer on this list: ffmpeg-skill/audio --audio-stream has existed since 0.9.1, and audio_stream
is implemented above on every operation that reads its input through audio.py.)
Repository guide for future sessions: CLAUDE.md; current / experimental / planned state:
docs/STATE.md; contract versioning: docs/contract.md.
Support
If this skill saves you time, you can help keep it maintained through GitHub Sponsors.
Issues and pull requests are just as welcome.
License
Reviews (0)
Sign in to leave a review.
Leave a reviewNo results found