AI Watermarks Reality Check

Seven portable agent skills for checking what AI provenance evidence exists, whether it verifies, what a publishing pipeline destroys, and what must still be disclosed.

The pack deliberately does not remove watermarks or Content Credentials. It makes provenance observable without pretending that one signal can prove authorship.

Why this exists

Anthropic says newly launched Claude models are gaining imperceptible text watermarks and supported image files can carry signed C2PA metadata. Its public detector and detailed text-watermark specification are not yet available. Meanwhile, common publishing actions such as resizing, screenshots, conversion, and social uploads can remove file metadata.

This pack gives teams a reproducible workflow:

inspect → verify → transform-test → privacy-audit → disclosure-check

Agent and model compatibility

The skills work with Claude Code, Codex, Gemini CLI, and other Agent Skills-compatible tools.

Use them to audit supported evidence in files and text from Claude, ChatGPT, Gemini, open models, image generators, or any other source. Coverage follows the evidence type rather than a vendor name. C2PA, metadata, disclosure records, pipeline survival, and hidden Unicode channels can be inspected wherever they appear.

There is no universal detector for every proprietary or secret-key watermark. When an official detector, configuration, or key is unavailable, the result remains UNVERIFIABLE, UNSUPPORTED, or UNKNOWN.

The seven skills

Install

npx skills add Neeeophytee/ai-watermarks-reality-check

The current skills installer requires Node.js 22.20 or newer. The installed
skills themselves require only Python 3.9 or newer.

Or copy one directory from skills/ into your agent's skill directory. Each skill folder is self-contained: the shared parsing core is vendored into every scripts/ directory, so a single folder works on its own.

As a local MCP server

For MCP-compatible clients, all seven analyzers can also be exposed as local stdio tools:

{
  "mcpServers": {
    "provenance": {
      "command": "python3",
      "args": ["/absolute/path/to/mcp/server.py"]
    }
  }
}

No hosted service is required. The MCP client starts python3 mcp/server.py on the user's machine and exchanges JSON-RPC messages with it through standard input and output. The server dispatches calls to the same read-only analyzers, reads local files, and returns structured results. Cryptographic verification uses the user's local c2patool installation. Remote-manifest access stays disabled unless the caller explicitly enables it.

The server supports two protocol eras. It implements the current stateless revision
2026-07-28 and still answers the legacy initialize handshake used by
2025-11-25 and earlier clients.

Quick start

Every script uses the Python standard library and requires Python 3.9 or newer. Cryptographic C2PA verification additionally requires the official c2patool 0.20.0 or newer.

Start with the front door, which answers all five questions in one pass:

python3 skills/audit-provenance/scripts/audit_provenance.py image.png \
  --c2patool /path/to/c2patool \
  --trust-anchors /path/to/policy.pem

The individual analyzers remain directly available:

python3 skills/inspect-content-provenance/scripts/inspect_file.py image.png

python3 skills/verify-content-credentials/scripts/verify_c2pa.py image.png

python3 skills/map-provenance-survival/scripts/map_survival.py \
  --original original.png \
  --derivatives-dir transformed-copies/ \
  --derivative social-download:platform-roundtrip=downloaded.jpg \
  --c2patool /path/to/c2patool \
  --report survival-report.html

python3 skills/audit-metadata-privacy/scripts/audit_metadata.py image.png

python3 skills/check-ai-transparency/scripts/check_transparency.py \
  examples/transparency-record.json

python3 skills/detect-text-watermark/scripts/detect_text_watermark.py draft.md

Scripts emit JSON to stdout and diagnostics to stderr. Remote-manifest fetching is disabled by default and requires --allow-network. A missing verifier, an unsupported verifier version, an inaccessible remote manifest, or an unpublished detector produces an explicit unknown state rather than a pass.

Survival reports refuse to overwrite an existing file. They show only filenames
by default, while the machine-readable JSON retains the full reproducibility
record. Use --include-paths only when the report will stay private. A generated
report never claims that a proprietary watermark detector ran.

See the generated example and the
community benchmark template before
publishing a result.

What's new in v0.2.0

The provenance survival skill can now audit a directory of edited or downloaded
copies in one run and create a self-contained Markdown or HTML report. This is
useful for testing a CMS, editor, CDN, social network, or watermark-removal
workflow without changing any of the files being measured.

• --derivatives-dir recursively adds visible, non-symlink files with stable labels.
• --report survival-report.html creates a portable, script-free report.
• Local paths and the original command are redacted from reports by default.
• The JSON result remains on stdout and continues to validate against the published schema.
• Every report states its boundary: this workflow measures C2PA survival, not proprietary pixel or keyed text watermarks.

Evidence model

POSSIBLE means a structural carrier, sidecar, or format-appropriate malformed
hint was located; inspect the marker confidence before acting. Only
STRUCTURAL means the location and carrier form match the specification.
PRESENT is only ever returned by a conforming verifier. ABSENT is emitted
only for a completed bounded scan of a supported container or an explicit live
verifier "no claim" result. VALID never automatically means TRUSTED.
Absence of a mark never proves that content was human-made.

A literal mention of "C2PA" in readable text is recorded in c2pa_mentions and is never evidence.

C2PA carriers checked

Detection is structural, at the locations the C2PA 2.4 specification defines.

Fail closed. The right-hand column is the important one. For formats marked
no, this build inspects the carriers listed but cannot walk the container
exhaustively because of compressed PDF object streams and fragmented BMFF.
Finding nothing therefore yields UNKNOWN with a stated reason, never ABSENT.

Partial scans

A bounded read can never produce a conclusive clean result. detect-text-watermark
streams the whole file where practical and always reports:

If scan_complete is false, the status is INCONCLUSIVE with a reason and exit
2, even when nothing suspicious was seen in the part that was read. A prefix
hash is never presented as the file hash.

Exit-code contract

Every entrypoint uses the same convention, so an orchestrating agent can apply one rule:

Output schemas

Machine-readable JSON Schemas for all seven tools live in schemas/. Every branch of every tool returns the same top-level field set, and CI validates real outputs against these schemas. The schemas also encode the evidence invariants: a document claiming integrity: VALID with manifest_presence: ABSENT fails validation.

Text watermark detection

detect-text-watermark runs a detector registry and reports honestly per adapter:

The state vocabulary distinguishes five situations, so a caller can never mistake
"did not look" for "looked and found nothing":

Only DETECTED and NOT_DETECTED mean a detector ran, and only those
contribute to the top-level status. SynthID and KGW accept configuration but
this build performs no scoring for either; they are unavailable
integrations, not silent passes.

Keyed model-level watermarks bias token sampling with a secret key; detection requires that key. A third party cannot check them, and this pack says so rather than guessing.

A standards-compliant C2PA text manifest (2.4 §A.8) is itself a run of variation
selectors after a ZWNBSP. It is recognised as provenance and excluded from
covert-channel findings rather than reported as suspicious.

The covert-channel scan finds Unicode tag characters (ASCII smuggling), bidi overrides, variation selectors, zero-width characters, exotic spaces, and mixed-script homoglyphs. These are text-integrity and prompt-injection signals, not watermark evidence: they identify no model or vendor.

Deliberate non-goal: statistical or stylometric AI-text classifiers are not implemented and will not be added. Their false-positive rates make them unsafe for accusing a person of AI authorship.

Scope and safety

• No watermark removal, evasion, or provenance stripping.
• No source files are modified.
• No authorship verdicts.
• No legal-compliance claims.
• No statistical AI-text detection.
• Reports avoid printing sensitive metadata values by default.

Sources

See SOURCES.md. Product behavior is volatile; the source review date is recorded there.

License

GNU Affero General Public License v3.0 (AGPL-3.0-only)