Revelio

Cost-efficient agentic memory-safety vulnerability detection for repository-scale codebases.

📄 Paper (arXiv) · UC Berkeley

Revelio is an end-to-end AI agent that discovers memory-safety vulnerabilities in large C/C++ codebases, and only reports a finding once it has reproduced it with a sanitizer. Because every discovery is backed by an executable Proof-of-Vulnerability (PoV), Revelio avoids the hallucinated, unverifiable reports that plague naive LLM-based bug hunting.

• Discovers candidate vulnerabilities through broad, high-recall code review with cheap models.
• Confirms each candidate by building a PoV input and running the program under AddressSanitizer / UBSan / MSan.
• Reports only sanitizer-confirmed bugs, with a full write-up and reproduction steps.

In the evaluation, Revelio uncovered 19 previously unknown memory-safety vulnerabilities (7 CVEs assigned/requested) across mature OSS-Fuzz projects that had already been continuously fuzzed for 5–8 years, at roughly $42 and ~1 hour per project with zero false positives. On the CyberGym/ARVO benchmark it substantially outperformed frontier coding agents (Claude Code, Codex, etc.) at comparable cost.

CVE Credit: CVE-2026-42216, CVE-2026-42217, CVE-2026-41071, CVE-2026-47254, CVE-2026-12725

Architecture

Revelio's key insight is to separate speculative reasoning from executable confirmation. Detection is decomposed into two stages: a cheap, high-recall Generation Stage and a high-precision Confirmation Stage. Cheap models cast a wide net (a false hypothesis can be filtered later); stronger models plus a deterministic sanitizer serve as the trustworthy verifier.

 repository ──► Stage 1: Hypothesis Generation ──► Stage 2: Hypothesis Confirmation ──► bug reports
 source code       (cheap, high recall)                (stronger, high precision)          + PoV files
                ScanFilterOrchestrator                MultiAgentOrchestrator

Stage 1 — Hypothesis Generation

Scans every source file to build a broad pool of candidate hypotheses, then funnels it down through triage, dedup, filtering, and ranking. Organized as two bands:

Proposal band — high-recall expansion (per file):

1. Static preprocessing — parse with tree-sitter, extract function boundaries, and run a lightweight intraprocedural check analyzer that flags which parameters lack NULL/bounds/validation guards.
2. Summarize file content — one LLM pass builds a functional summary used as context (no hypotheses yet).
3. Synthesize hypotheses — multiple LLM passes from different angles (whole-file, focused passes, per-function batches) emit candidate hypotheses as structured JSON → Raw Hypothesis Pool.

Refinement band — precision and ranking:

1. Sanitizer-aware triage — one LLM call per hypothesis hints whether it is a real, sanitizer-triggerable memory-safety bug (asan/ubsan/msan) and tags suggested severity/primitive/CWE → In-scope Hypotheses.
2. Deduplicate root causes — line/CWE-overlap prefilter + pairwise LLM judgement + union-find merge → Merged Hypotheses.
3. Reachability annotation — arvo targets <symbol> (deterministic nm lookup) records which test-harness (entry-point) binaries link each hotspot function. Advisory ranking signal, not a hard filter.
4. Independent static filtering — a real Docker sub-agent (DefaultAgent + unrestricted bash) traces callers/callees repo-wide and votes VALID/INVALID on each hypothesis.
5. Rank for PoV confirmation — deterministic sort by (reachable, severity, confidence) → Ranked Hypothesis Queue.

The queue is saved to hypotheses.json (reusable via --hypotheses-file).

Stage 2 — Hypothesis Confirmation by PoV Construction

For each of the top --top-n hypotheses:

1. Iterative PoV construction (PoVBuilderAgent) — a stronger model receives a context packet (hypothesis, code references, selected harness, etc.) and iterates with three tools: bash (inspect code / write files), validate (test a candidate PoV), and finish (submit results). It writes a Python script that emits raw PoV input bytes and refines it based on validation feedback, up to --max-pov-attempts times.
2. Sanitizer-backed confirmation — the validate tool runs the harness against all available sanitizers (asan, ubsan, msan) and parses crash signatures. A hypothesis is confirmed only when a sanitizer reproducibly crashes; the orchestrator independently re-executes the input to guard against fabricated success.
3. Reporting (ReporterAgent) — writes the final developer-facing bug report with vulnerability details, triggering input, sanitizer output, and reproduction steps.

Deliverables (report, PoV input, PoV generator script) are copied out of the container only on a confirmed crash.

Model tiers

Revelio assigns different model tiers to different tasks:

Quick Start

Requirements

• Python ≥ 3.10 (3.12 recommended)
• Docker (or Podman) — every codebase-under-test runs inside a container for isolation and reproducibility

Installation

git clone https://github.com/m1-llie/Revelio.git
cd Revelio

conda create -n revelio python=3.12 -y
conda activate revelio
pip install -e .

Installing the package also exposes a revelio console command (equivalent to python -m revelio.run.detect), used throughout this README.

Configuration

Create a .env file in the project root (or export the equivalent environment variables):

# Required: a default model + its API key
MODEL_NAME=anthropic/claude-haiku-4-5
ANTHROPIC_API_KEY=sk-ant-...

# Or use Gemini
# MODEL_NAME=gemini/gemini-2.5-pro
# GEMINI_API_KEY=AIza...

MODEL_NAME is the default for --model; the provider prefix (anthropic/, gemini/, …) is required. See Environment Variables for the full list.

Building OSS-Fuzz Docker Images

scripts/prepare_ossfuzz_project.sh builds any OSS-Fuzz project into a revelio-ready Docker image with the same interface as ARVO images.

scripts/prepare_ossfuzz_project.sh openssl                           # single project
scripts/prepare_ossfuzz_project.sh openssl assimp curl               # multiple projects
scripts/prepare_ossfuzz_project.sh --sanitizers asan,ubsan openssl   # if skip MSan
scripts/prepare_ossfuzz_project.sh --oss-fuzz-dir /data/oss-fuzz openssl

The script clones/updates oss-fuzz, builds fuzzers with each sanitizer via infra/helper.py build_fuzzers, packages them into a single Docker image, and cleans it for zero-day detection. Sanitizers that fail to build are skipped automatically.

The resulting revelio/<project>:latest image contains:

/src/<project>/              source code (from the gcr.io/oss-fuzz/<project> base image)
/out/asan/<fuzzer_binaries>  AddressSanitizer builds
/out/ubsan/<fuzzer_binaries> UndefinedBehaviorSanitizer builds
/out/msan/<fuzzer_binaries>  MemorySanitizer builds
/usr/bin/arvo                fuzzer runner script (scripts/arvo_ossfuzz)

Re-run the script to pick up upstream source code changes.

The arvo script

scripts/arvo_ossfuzz is installed at /usr/bin/arvo inside the container. It provides the same interface as ARVO images, extended for multi-sanitizer and multi-target support:

arvo                         # run default fuzzer with /tmp/poc (SANITIZER=asan)
arvo list                    # list fuzz targets for current SANITIZER
arvo list --all              # list fuzz targets across all sanitizers
arvo run <fuzzer> [poc]      # run a specific fuzzer (default input: /tmp/poc)
arvo targets <symbol>        # find which fuzz targets link a given function (nm lookup)
arvo compile                 # recompile the project
SANITIZER=ubsan arvo         # switch sanitizer

SANITIZER env var (default: asan) selects which /out/<sanitizer>/ build to use. DEFAULT_FUZZER env var overrides auto-detection when multiple targets exist. Standard ASAN/MSAN/UBSAN env vars are exported automatically.

The orchestrator uses arvo targets <function> to match hypotheses to reachable fuzz targets before launching the PoV builder. The validate tool uses SANITIZER= to test PoVs against all available sanitizers.

Verifying the image

docker run --rm revelio/openssl:latest arvo list --all
docker run --rm -v /path/to/pov:/tmp/poc:ro revelio/openssl:latest arvo run openssl_fuzzer
docker run --rm -v /path/to/pov:/tmp/poc:ro -e SANITIZER=ubsan revelio/openssl:latest arvo run openssl_fuzzer

Revelio for Vulnerability Detection

ARVO Targets

ARVO provides pre-built Docker images with fuzzing infrastructure.

revelio \
    --arvo n132/arvo:42470801-vul \
    --hypothesis-model anthropic/claude-haiku-4-5

# Scan a specific file only
revelio \
    --arvo n132/arvo:42470801-vul \
    --hypothesis-model anthropic/claude-haiku-4-5 \
    --target-file ffmpeg/tools/target_dec_fuzzer.c

Note: arvo:xxx-vul-clean images are produced using python -m revelio.run.clean_arvo
to remove pre-existing PoVs, crashers, and seed corpus from ARVO-vul images.

OSS-Fuzz Targets

For OSS-Fuzz Docker images built with the steps above:

revelio \
    --arvo revelio/assimp:latest \
    --hypothesis-model anthropic/claude-haiku-4-5

Custom Projects

revelio \
    --project ./my-project \
    --hypothesis-model anthropic/claude-haiku-4-5 \
    --target-file src/parser.c

Use --docker-image to specify a custom Docker base image (default: revelio/memcheck:latest).

Resuming from saved hypotheses

Stage 1 saves its ranked queue to output/<run_id>/hypotheses.json. You can skip the hypothesis generation stage and go straight to Stage 2 PoV construction:

revelio \
    --arvo revelio/assimp:latest \
    --hypotheses-file output/<run_id>/hypotheses.json \
    --pov-model anthropic/claude-sonnet-4-6

Key Flags

Environment Variables

Model Selection

API Keys

Tip: You only need the provider-specific key for your chosen model. MODEL_API_KEY is an alternative that works across providers via litellm.

Cost and Rate Limits

Runtime

Outputs

Each run writes to output/<run_id>/:

output/<run_id>/
├── manifest.json          # Run metadata (target, model, pipeline mode, parameters)
├── events.jsonl           # Append-only event log (JSON Lines)
├── log.txt                # Human-readable timestamped log
├── hypotheses.json        # Saved hypotheses (reusable with --hypotheses-file)
├── trajectory.json        # Aggregated multi-agent trajectories
└── artifacts/
    ├── handoffs/          # Inter-agent data (one JSON per stage + hypothesis)
    │   ├── hypotheses.json
    │   ├── pov_recipe_H01.json
    │   ├── validation_H01.json
    │   └── report_H01.json
    └── deliverables/      # Final outputs (copied from container on success)
        ├── final_report_H01.md
        ├── pov_H01
        └── result_script_H01.py

• manifest.json — records run_id, target, model, pipeline mode, top_n, max_workers
• events.jsonl — real-time event stream: run_start, agent_start, agent_end, pov_attempt, validation_failed, run_success, etc.
• trajectory.json — full conversation history for all agents, keyed by agent name
• handoffs/ — structured JSON data passed between pipeline stages
• deliverables/ — the final artifacts: bug report, PoV input file, and PoV generator script (only present when a vulnerability is confirmed)

Cost Reports

# Summarize direct API cost plus cache/uncached token-cost estimates
SILENT_STARTUP=1 python tools/cost_report.py output/<run_id>

# Also write output/<run_id>/cost_report.json and cost_report.md
SILENT_STARTUP=1 python tools/cost_report.py output/<run_id> --write

The report reads saved trajectory.json and traces/*.json. For agent trajectories it can split cost into cached input, uncached input, and output when Anthropic/LiteLLM usage fields are present. Some scan_filter proposal/refinement traces currently only save aggregate cost/calls, so those stages are included in direct API cost but cannot always be split into cached/uncached tokens retroactively.

Validating PoVs

Validate PoVs against ARVO -fix images:

python tools/validate_pov.py \
    --run-dir output/<run_id> \
    --pov output/<run_id>/artifacts/deliverables/pov_H01

Project Structure

Revelio/
├── src/revelio/
│   ├── agents/          # Agent implementations (DefaultAgent)
│   ├── analysis/        # Static C/C++ check analyzer (tree-sitter)
│   ├── artifacts/       # Artifact store (typed, append-only, thread-safe)
│   ├── config/          # YAML configs: agents/, arvo_targets.json
│   ├── environments/    # Execution environments (Docker)
│   ├── models/          # LLM interfaces (LiteLLM, Anthropic)
│   ├── orchestrator/    # ScanFilterOrchestrator (Stage 1) + MultiAgentOrchestrator (Stage 2)
│   ├── run/             # CLI entry point (detect.py) + clean_arvo
│   ├── tools/           # Agent tools (bash/validate/finish)
│   └── utils/           # Utility functions
├── tools/               # Standalone CLIs: cost_report, inspector, validate_pov
├── scripts/             # OSS-Fuzz image builder, arvo runner, batch helpers
├── docker/              # Base image Dockerfiles
└── website/             # Web trace viewer

Check Agent Running Trajectories

Textual TUI for browsing trajectory JSON files

# TUI trajectory inspector (vim-style keybindings)
python tools/inspector.py output/<run_id>/trajectory.json

# Or browse an entire output directory
python tools/inspector.py output/

Web Trace Viewer

Browse agent run outputs in a web interface:

python3 website/server.py [--host HOST] [--port PORT] [--output-dir DIR ...]

Remote access via SSH tunnel:

# On local machine
ssh -L 8877:127.0.0.1:8877 <username>@<server>

# Then open http://localhost:8877