color-grading-skill

Deterministic colour grading execution for AI video agents.
Executes colour operations an AI agent has already decided on as a typed, deterministic operation graph on top of ffmpeg-skill.
Typed operations · Validated artifacts · Full provenance · No LLM, no prompt, no automatic LUT or "look" selection.

Quick start · How it works · Supported operations · Docs

git clone https://github.com/kajisho5/color-grading-skill && cd color-grading-skill && pip install -e .

color-grading-skill executes colour operations that an AI agent has already decided on — HDR → SDR tone mapping,
3D LUT application, colour-tag retagging, Dolby Vision RPU removal — as a typed, deterministic operation graph on
top of ffmpeg-skill, with every output re-probed and validated and every
result carrying full provenance.

It does not decide anything. No LLM, no prompt, no reasoning, no automatic LUT or "look" selection, no judging
whether a frame looks cinematic. It takes a typed request, executes exactly what it says, refuses everything else,
and reports what it observed.

Contents
What it does · Quick start · How it works ·
Supported operations · Deterministic execution ·
Artifact validation · Provenance · Security ·
Built for agents · Errors · Limitations ·
Testing · Docs · Ecosystem · Support

What it is, and what it is not

That split is the whole design: video-production-agent
decides what to do and builds the request; color-grading-skill decides how to execute it safely and
deterministically; ffmpeg-skill is the media execution engine that
actually runs FFmpeg. Full boundary, including how this differs from media-analysis-skill's pure measurement role:
docs/architecture.md.

Quick start

Requirements: Python 3.9+ (standard library only), an
ffmpeg-skill checkout of any version that has the flags this skill
emits (0.12.3 and every later release, 2.x included; there is no version window, doctor reports what it
detected), and ffmpeg / ffprobe on PATH for ffmpeg-skill itself.

# 1. install this skill
pip install -e .

# 2. check the machine: ffmpeg-skill, ffmpeg/ffprobe, and every capability the five operations need
color-grading doctor --json --ffmpeg-skill /path/to/ffmpeg-skill

# 3. read the machine-readable contract (for agent frameworks)
color-grading skill --json | head -40

Write a request naming the operation graph — here, HDR tone mapping followed by a LUT:

{
  "schema": "color-grading/request@1",
  "project": {
    "project_id": "grade-42",
    "source": {"source_id": "raw", "path": "footage/iphone_hdr.mov"},
    "operations": [
      {"op_id": "sdr", "type": "HDR_TO_SDR", "input": "source", "parameters": {"tonemap": "hable"}},
      {"op_id": "grade", "type": "LUT_APPLY", "input": "op:sdr", "parameters": {"lut_path": "luts/look_a.cube"}}
    ],
    "outputs": [{"output_id": "graded", "operation": "op:grade", "path": "deliver/grade-42.mov", "format": "mov"}]
  }
}

Then plan (read-only) and run it:

color-grading plan request.json --json     # graph, tool selection, expected geometry — writes no media
color-grading run request.json --json      # execute; stdout: exactly one color-grading/response@1 document

ok: true and outputs[0].status: "completed" mean the artifact exists at deliver/grade-42.mov, was re-probed,
and matched what HDR_TO_SDR and LUT_APPLY are each supposed to have measurably done. Full request/response
schema: docs/architecture.md; the plain-language contract an agent reads: SKILL.md.

How it works

flowchart LR
    A["video-production-agent<br/>decides WHAT<br/>which LUT · which tonemap curve · which retag target"] -->|"color-grading/request@1"| B["color-grading-skill<br/>decides HOW<br/>typed, deterministic execution"]
    B -->|"python3 scripts/probe.py, color.py"| C["ffmpeg-skill<br/>media execution engine"]
    C --> D[FFmpeg / FFprobe]

flowchart TD
    R["JSON request<br/>color-grading/request@1"] --> V["Validation<br/>typed schema · forbidden fields · PathPolicy"]
    V --> G["Operation Graph<br/>deterministic topological order"]
    G --> I["Deterministic identity<br/>sha256 of type + parameters + input + tool versions"]
    I --> E["Execute via ffmpeg-skill/color<br/>one intermediate per node, reused when identity matches"]
    E --> C2["Artifact validation<br/>re-probe: colour tags · HDR state · Dolby Vision · geometry"]
    C2 --> O["JSON response<br/>color-grading/response@1 + provenance"]

Nothing in this pipeline reasons about the image. Every box is a typed, mechanical step; the only judgement call
(which operation, which LUT, which curve) already happened upstream, in the request this skill receives.

Supported operations

Each maps 1:1 onto one mode of ffmpeg-skill/color; every operation takes exactly one input and none of them
change duration or frame geometry (verified against the source within a documented tolerance). Full parameter
tables, defaults, ranges and the exact ffmpeg-skill flags behind each one: docs/architecture.md
and docs/ffmpeg-skill.md.

Output containers (mp4, mov, m4v, mkv) must match the source's — this skill does not convert containers
(use ffmpeg-skill/export for that).

Deterministic execution

Every operation gets an identity — sha256 over canonical JSON of {type, effective parameters, input identity, tool versions} — computed before anything runs. Two things follow from that:

• Reuse. Re-running the same request with the same inputs and the same ffmpeg-skill/FFmpeg versions reuses every
  matching intermediate instead of re-encoding. A tampered or truncated intermediate is detected (size + sha256
  re-checked against the manifest) and re-processed, never falsely trusted.
• Content-addressed LUTs. LUT_APPLY's identity uses the LUT file's own sha256, never its path: a LUT moved to
  a new location keeps its identity; a LUT edited in place at the same path gets a new one. A LUT is data, resolved
  through its own path policy and hashed — never a filter string.

No timestamps, no UUIDs, stable topological order. Identical request + identical inputs + identical tool versions
→ identical plan_id and operation ids, every time. Design rationale: docs/decisions.md.

Artifact validation

This skill does not trust a 0 exit code from ffmpeg-skill as proof that an operation worked. Every artifact is
re-probed and checked: exists, size > 0, readable, has a video stream, duration and resolution unchanged from the
source — plus the operation's own measurable effect:

This caught a real ffmpeg-skill/FFmpeg limitation during development: a stream-copy retag can exit 0 without
actually rewriting colour tags already baked into a source's bitstream. This skill reports that as
VALIDATION_ERROR, never a false success — see docs/ffmpeg-skill.md for the measured case.

Provenance

Every result carries what it did, not just that it succeeded: operation_id, type, tool, tool_versions,
input_hash, output_hash, the LUT's own hash (for LUT_APPLY), measurements, and tool_commands_observed (the
commands ffmpeg-skill itself reports having run, not a description this skill reconstructs afterwards). Every
output carries the full operation chain back to the source, each link with its own hash and status. Measured colour
metadata is always observed (read back from ffmpeg-skill/probe), never inferred from the request.

Security

• No shell, no eval, no user-supplied command, argv, executable, script or filter string — those field names are
  rejected anywhere in the request document, at any nesting depth; unknown fields are rejected everywhere.
• Every value that reaches a subprocess argv is a fixed flag, a number formatted by this skill, an enum already
  validated against the schema, or a resolved absolute path — never a raw string from the request.
• A LUT is resolved through its own path policy, separate from the video input's (a caller allowed to read footage
  under /footage cannot thereby name any file on the machine as a LUT), checked for a .cube extension and a size
  ceiling, and hashed — never parsed or treated as a filter string.
• Inputs and LUTs must be regular files (symlinks resolved before every check); outputs must resolve inside the
  workspace — no .., no symlinked escape, no reserved Windows device names — and may never be an existing input
  file unless overwrite: true is explicit.
• Only ffmpeg-skill/probe and ffmpeg-skill/color can ever be started, from a checkout named by the caller, never
  by the request document, with a minimal child environment.

Full enforcement table, including what is deliberately not enforced (e.g. no root restriction by default, matching
the rest of the ecosystem): docs/security.md.

Built for agents

Machine-readable contract

color-grading skill --json      # color-grading/contract@1: operations, parameters, capabilities, errors, schemas
color-grading doctor --json     # color-grading/doctor@1: environment vs. contract, per-capability status

The contract states stable identifiers (skill_id, operations[].type, operations[].parameters,
operation_aliases, unsupported_operations, output_formats, errors.codes), all derived from the tables the code
runs on — there are no placeholder operations. For the AI Video Production OS registry it also carries
contract_version (1, this contract's schema version) and dependencies: ffmpeg-skill with
version_range: ">=1.0 <3.0" over ffmpeg-skill's own contract_version (range_of: "contract_version"), plus the
tools used (probe, color). That range is a registry declaration; whether a given checkout actually works is still
decided flag by flag (ADR-19). doctor reports every capability (ffmpeg-skill, ffmpeg, ffprobe,
filter:<name>, encoder:<name>, bsf:<name>) as supported, unsupported or unknown — an installed capability
is never reported absent, and a failed detection is never silently read as a pass.

provides lists this Skill's four operations by their cross-repository Capability id (color.hdr_to_sdr,
color.lut_apply, color.retag, color.strip_dovi), each with the operation it maps to, its tool_id (always
color-grading/run, since this Skill exposes one execution tool), and a lifecycle. This is a different vocabulary
from the doctor/capabilities one above (environment/binary detection, e.g. ffmpeg, filter:<name>): provides
is for kajisho5/AI-video-production-OS's CapabilityContract.provides (see that project's docs/SPEC.md), so a
registry can resolve "who provides color.hdr_to_sdr" without hardcoding this repository. It is additive and
derived from model.OPERATION_TYPES; see docs/decisions.md ADR-15.

CLI

stdout under --json is always exactly one document, on success and failure alike; stderr carries diagnostics.
Exit codes are stable per error code (see Error handling below); every command supports --help
for its full flag list.

ffmpeg-skill integration

This skill never calls ffmpeg or ffprobe itself. Every process it starts is
python3 <ffmpeg-skill>/scripts/{probe,color}.py with a typed argv, verified against ffmpeg-skill's own
machine-readable contract (contract_version 1.0 in ffmpeg-skill 1.x/2.x, 2.0 in 3.x) before use. Compatibility is detected, not looked up by version
number: every flag this skill emits must be present in the located checkout's contract (or --help), the CRF flag is
whichever of --quality (ffmpeg-skill ≥ 1.10) / --crf (before 2.0) it offers, and probe's HDR flag is read as
bt2020_or_hdr where it exists (2.0 narrowed hdr). A checkout missing a flag is refused with that flag named.
Exact tools, flags and every measured compatibility gap: docs/ffmpeg-skill.md.

Error handling

Every code has a stable exit code (errors.exit_codes in the contract); ok mirrors the process exit code.

Current limitations

Stated as boundary, not as bugs to be worked around with a private filter string:

• White balance is an alias, not its own operation type: WHITE_BALANCE is accepted in a request and resolved
  to PRIMARY_CORRECTION with only temperature/tint set (any other correction parameter under it is
  INVALID_REQUEST); it is listed in the contract's operation_aliases, not in operations[], and results and
  provenance report PRIMARY_CORRECTION (ADR-20). ffmpeg-skill has no single "white balance" flag, and this skill
  does not need one: the two typed flags that achieve it already exist. There is no automatic white balance (no
  grey-world/neutral-point estimation): the caller supplies the values. (Gamma, lift, gain, levels and curves are
  implemented too, as eight more optional PRIMARY_CORRECTION parameters, since ffmpeg-skill 0.12.3 added typed
  --correct flags for them.)
  Details per operation: docs/ffmpeg-skill.md.
• One source per request; no MIX/CONCAT analogue for colour.
• No container/format conversion — output keeps the source's container.
• A chain of N operations costs N re-encodes/stream-copies (ffmpeg-skill/color's mode flags are mutually exclusive).
• LUT_APPLY's lut_strength of exactly 0.0 behaves like 1.0 (full LUT) — ffmpeg-skill's own documented
  behaviour, not something this skill introduces or can fix on its own.
• No cache eviction: <workspace>/.color-grading/<project_id>/ grows until the caller removes it.

Testing

pip install -e . pytest
export COLOR_GRADING_FFMPEG_SKILL_DIR=/path/to/ffmpeg-skill     # or clone it as ../ffmpeg-skill
python -m pytest -q

284 tests, nothing skipped by default, nothing mocked in the integration layer: 152 unit (schema, graph, path
policy, determinism), 49 security (injection, path/symlink escapes, argv audit), 5 contract (contract ⇔
implementation, doctor), 29 ffmpeg-skill compatibility (capability detection across old/new layouts and contracts), and 49 integration tests that run every operation against real video through a real
ffmpeg-skill checkout and real FFmpeg — including deterministic pixel-level checks for LUT_APPLY and
PRIMARY_CORRECTION (including its gamma/lift/gain/levels_*/curves parameters), PRIMARY_CORRECTION's
observed before/after measurements, and a reproduction of the
exit-0-but-unchanged retag case above. CI (.github/workflows/tests.yml) runs Linux (Python 3.9, 3.11), Windows and
macOS, each against a real FFmpeg install and ffmpeg-skill main (also daily), plus one Linux job against the oldest
supported ffmpeg-skill commit (0.12.3). File-by-file coverage and what
real-media verification does and does not prove: docs/testing.md.

Docs

Relationship to the other skills

audio-production-skill, video-editing-skill, transcription-skill, subtitle-skill and QC are not touched by
this skill — it has no audio, cut/trim, speech or final-QC role.

Support

If this skill saves you time, you can help keep it maintained through GitHub Sponsors.
Issues and pull requests are just as welcome.

License

MIT