tiktest

automated tiktok-style video review of every pr.

the more i shipped stuff with claude the more i realised; in the long term the most value i can add as a human is testing - finding the stuff other humans would struggle with...

...but testing is boring. so i tried to make it less boring - or at least take way less of my time.

Play with sound to hear narration and watch normal speed

Get it running in a few mins

Copy one of these prompts into Claude Code. Claude does the wiring.

Wire it into a GitHub repo — cd into your repo, open Claude Code, paste:

Set up tik-test (https://github.com/marcushyett/tik-test — 45s video reviews on every PR) on this repo. Goal: working tik-test.yml + drafted tiktest.md + secret instructions, then test on a real PR.
1. Pick a template (local-dev / vercel-preview / staging-with-services) from inspecting package.json, framework configs, and deploy setup; ask if ambiguous. Create .github/workflows/tik-test.yml from it, adapted (build cmd, dev port, working-directory).
2. Draft tiktest.md at repo root: read README + package.json + 5-10 representative source files to infer flows; detect login (ask me for creds if there's a wall); list 3-5 highest-risk user-facing surfaces. Format: one-paragraph app description → login → "Risky surfaces" bullets → optional "Hints" with test-IDs. Show me the draft and wait for confirmation before committing. Warn if CLAUDE.md / AGENTS.md sit alongside it (silent-fallback footgun).
3. List the secrets I need to add: CLAUDE_CODE_OAUTH_TOKEN (`claude setup-token`, required) plus optionals you detect apply (OPENAI_API_KEY for narration; VERCEL_AUTOMATION_BYPASS_SECRET if Vercel Deployment Protection is on). One line each on where to get them. Mention that TIKTEST_OWNERS in repo Variables can restrict who triggers runs (default = any collaborator).
4. Open the PR. Heads-up: its diff has no UI surface, so the auto-review will skip with a "no UI changed" comment — expected, not a failure.
5. After I merge it, propose a small visible UI change (one-screen nit from open issues or your own suggestion) to be the real first test PR. Wait for my okay, then ship on a separate branch.

Install the Claude Code plugin — records a walkthrough of whatever you just shipped, locally.

Slash commands have to be typed by you (Claude can't invoke them from inside a prompt). Open Claude Code in your project and run these three yourself, one at a time:

/plugin marketplace add marcushyett/tik-test
/plugin install tiktest@tiktest
/reload-plugins

Then paste this prompt so Claude finishes the wiring:

The tik-test Claude Code plugin is now installed. Finish setting it up for this project:
1. Verify the plugin's runtime prereqs are present and fix what's missing for my OS:
   - `ffmpeg` on PATH (`brew install ffmpeg` on macOS, `sudo apt install ffmpeg` on Linux)
   - Playwright Chromium (`npx playwright install chromium`)
   - `tik-test` CLI globally (`npm i -g tik-test`)
2. If a `tiktest.md` already exists at repo root, summarise it and ask me whether to keep, expand, or replace it before doing anything destructive. Otherwise run `/tiktest:setup` and ask me for anything you can't infer (project name, dev-server start command, login/auth flow, key user paths to record).
3. Tell me how to invoke `/tiktest:run` (full video, MP4 to Desktop) and `/tiktest:quick` (checklist only, no video) when I ship a feature, including the optional URL / focus-hint argument forms.

The problem

AI ships 20+ PRs a day that mostly work. Where it fails is taste: awkward flows, off-brand copy, the counter-intuitive ways real users click. Catching that means dropping what you're doing, opening the PR, exercising it, then switching back. Twenty context switches a day.

tik-test puts a 45-60s narrated video on every PR: happy path, edge cases, bugs called out on camera. You skim 20 in your inbox between meetings and only context-switch into the ones that look off.

The bottleneck of building products is shifting from writing code to testing it well. tik-test is a bet on that.

Prerequisites

Optional:

• OPENAI_API_KEY for voice narration (silent on Linux without it).
• VERCEL_AUTOMATION_BYPASS_SECRET for protected previews.
• Claude Code CLI (only required if you want to use the Claude Code plugin): install from https://docs.claude.com/en/docs/claude-code/setup.

Telling tik-test how to test your app

Two pieces of context, in two different places:

1. Project-level: tiktest.md at your repo root. Stable across PRs. Describes the app and how to sign in. Free-form prose, no schema.

# Acme

Acme is a project tracker for engineering teams.

Login: email `[email protected]`, password `hunter2`

2. PR-level: a "what to test" note in your PR description. Changes per PR. Tells the agent which surfaces this change touches and what's risky. Skip it and tik-test plans from the diff alone (best-effort).

## What to test
- Bulk archive on the tasks list. Select 5+ items, archive, confirm count.
- Empty state when all tasks are archived.

Where the URL comes from

Two ways tik-test learns where to point the browser. Pick whichever fits your setup:

• Per-PR preview URLs (Vercel / Netlify / similar). Auto-detected from the deployment_status event in CI. Don't put a URL in tiktest.md; the action handles it. This is the common case.
• Stable test-environment URL (e.g. dev.acme.app always points at trunk). Add it to tiktest.md: any https://... link in the file works.

If both are present the auto-detected preview URL wins.

How it's designed (and what it won't suit)

tik-test was built for small, focused PRs that touch one user-facing slice at a time. This is what Alistair Cockburn calls Elephant Carpaccio: nano-slices that each ship value end-to-end. If you bundle ten features into a 2,000-line PR, break it up, or the video will skim two and miss the rest.

Works well

• PRs touching one user-facing slice. Forms, dialogs, lists, navigation, a single end-to-end flow. The agent picks 1-3 goals and probes them deeply.
• Flows the agent can drive via Playwright MCP. Clicks, typing, hovers, keyboard shortcuts, dropdowns, file uploads. Slow flows with login spinners are fine; the editor crops idle waits to 0.3s.
• Multi-page wizards with seconds of actual interaction. Page transitions aren't the problem; total interaction count is.
• Public-ish preview URLs. Vercel-protected works via the bypass secret.
• Visible failures. Broken layout, validation that contradicts the input, button that doesn't fire, badge with the wrong colour.

Won't suit

• Giant PRs adding 10+ features at once. The plan picks 1-3 goals; the rest go uncovered.
• Design studios, canvas tools, drag-precise interactions. Figma-style apps, video editors. Playwright can drive a <canvas> but the agent can't tell whether it dragged the handle to the right spot.
• Apps gated behind SAML, SSO, MFA, or per-tenant subdomains with no automation bypass.
• Subtle regressions with no visible surface. Wrong analytics event, wrong DB write, p95 perf regression, Safari-14-only CSS bug.

Defaults reflect how I review PRs: small slices, fast feedback, video-first. Run node dist/cli.js config to retune.

In the wild

The demo above is the bundled Taskpad app. Agent caught two planted bugs (case-sensitive search, priority-sort reversed) and posted a request-changes review.

Self-test: every PR to this repo gets reviewed by tik-test against the bundled Taskpad demo — see the most recent PR for an example video.

Watch in a feed

tiktest.dev is a TikTok-style web viewer for your tik-test videos. Pure pass-through: GitHub is the backend, we store nothing, no auth required beyond your existing GitHub login. Self-host instead with node dist/cli.js view against your local runs directory.

How it works

                    ┌──────────────┐
                    │  tiktest.md  │◄── PR body, config, optional inline goal list
                    └──────┬───────┘
                           ▼
     ┌─────────────────────────────────────────────┐
     │ 1. Plan           1-3 high-level goals      │  ← Claude generates
     │ 2. Drive          Agent runs Playwright MCP │  ← records raw.webm + tool log
     │ 3. Narrate        Claude writes the script  │  ← intro + beats + outro, one call
     │ 4. Voice          OpenAI TTS (voice: ash)   │  ← one WAV per beat
     │ 5. Trim           FFmpeg crops idle waits   │  ← login spinners → 0.3s
     │ 6. Compose        Remotion overlays         │  ← pan/zoom, captions, clicks, audio
     │ 7. Checklist      Claude summarises 6-10    │  ← AI-checks list shown on outro
     │ 8. Publish        GitHub release + comment  │  ← + formal PR review (approve/reject)
     └─────────────────────────────────────────────┘
                           ▼
                  `highlights.mp4` + `preview.gif`

Design notes:

• One continuous master video, not per-step clips. Every overlay references the same timeline so desync is structurally impossible.
• Voice paces the video. Each beat sized to its narration; playbackRate clamped 1.0–1.6x.
• Remotion is compositor-only. CSS transforms over the pre-trimmed master; ~20 fps render on Apple Silicon in --quick mode.
• claude CLI invoked directly (not the SDK), so compute bills against your Claude Code Max subscription.

Install

# macOS
brew install ffmpeg node
gh auth login
git clone https://github.com/marcushyett/tik-test
cd tik-test
npm install
npx playwright install chromium
npm run build

# Ubuntu / GitHub Action runner
sudo apt-get install -y ffmpeg fonts-dejavu
npm install
npx playwright install chromium --with-deps
npm run build

Quickstart

1. Local app

python3 -m http.server 4173 --directory examples/todo-app &
node dist/cli.js run --config examples/todo-app/tiktest.md --quick

2. GitHub PR

export OPENAI_API_KEY=...                            # voice (optional)
export VERCEL_AUTOMATION_BYPASS_SECRET=...           # protected previews (optional)

node dist/cli.js pr https://github.com/owner/repo/pull/42

Useful flags:

3. Claude Code plugin

Already inside a Claude Code session? Four-step setup, two of which are one-time-per-machine.

Step 1 — install runtime prereqs (one-time per machine):

npm install -g tik-test                          # CLI binary the plugin shells out to
npx playwright install chromium                  # browser the agent drives

You also need ffmpeg on PATH (brew install ffmpeg on macOS, sudo apt install ffmpeg on Linux) and a claude CLI signed in (claude setup-token).

Step 2 — add the marketplace and install the plugin (one-time per machine):

In your Claude Code prompt:

/plugin marketplace add marcushyett/tik-test
/plugin install tiktest@tiktest

The slash commands are now available across all your future Claude Code sessions.

Step 3 — scaffold a tiktest.md for your project (one-time per project):

cd into your project, then:

/tiktest:setup

This inspects your package.json / README.md / framework configs to draft a tiktest.md, then asks you for anything it can't infer (login credentials, special URLs, areas to focus on).

Step 4 — record or test (whenever you ship a feature):

/tiktest:run                          # agent test pass + MP4 walkthrough on Desktop + pass/fail checklist
/tiktest:run http://localhost:5173    # explicit URL (skip the dev-server probe)
/tiktest:quick                        # no video — faster, prints checklist in chat

Or invoke the bundled sub-agent from any session: "Use the tiktest-runner agent to record a walkthrough of the feature I just shipped."

The agent reads git diff origin/main..HEAD, summarises what you changed, lists the things it'll exercise, then runs. Soft confirmation — interrupt with words if you want a different focus.

Updating the plugin later: /plugin marketplace update tiktest.

Hacking on the plugin itself? Clone the repo and use claude --plugin-dir ./plugin — see docs/PLUGIN.md for the development install + troubleshooting guide.

Inline test plan (optional)

By default the agent generates a plan from the PR diff plus your ## TikTest section. For deterministic coverage, add a ### Test Plan sub-section with a JSON goals list:

{
  "name": "Theater mode",
  "summary": "Verify the new full-screen viewer keeps keyboard shortcuts working.",
  "startUrl": "https://acme-pr-42.vercel.app",
  "goals": [
    { "id": "open-theater", "intent": "Open the Inspiration grid and click Theater on any card.", "shortLabel": "Open Theater" },
    { "id": "kbd-shortcuts", "intent": "While in Theater mode, press Down, S, then Esc, verifying each works.", "shortLabel": "Keyboard nav", "importance": "high" }
  ]
}

Goal fields: intent (required) · shortLabel (3-5 word checklist headline) · success (observable success condition) · importance (low · normal · high · critical).

Discovery order when running tik-test pr: tiktest.md (preferred), then README.md with a ## TikTest heading, then legacy fallbacks (claude.md → CLAUDE.md → .claude/claude.md → bare README.md).

GitHub Action

Quick start: copy one of the drop-in templates from
templates/workflows/ into your repo at
.github/workflows/tik-test.yml. There are three, picked by what your
app needs to boot:

• local-dev.yml — npm run dev
  inside the runner. Best for SPAs, static sites, Next.js with mocked
  APIs.
• vercel-preview.yml —
  drive your per-PR Vercel preview deployment. Includes the
  automation-bypass plumbing.
• staging-with-services.yml
  — boots Postgres + Redis service containers, runs migrations + seeds,
  then starts the app. Best for Rails / Django / Next.js + Prisma.

See templates/workflows/README.md
for the chooser tree and the per-template tiktest.md shape they expect.

The minimal hand-written form below is kept as a reference if you want
to wire it up yourself.

# .github/workflows/tik-test.yml (rename to whatever you want — pick a path
# that matches your repo layout)
name: tik-test review
on:
  deployment_status:                    # after Vercel/Netlify preview is Ready
  pull_request:                         # or run directly on PRs
  workflow_dispatch:
    inputs:
      pr_number:
        description: PR number to review
        required: true

permissions:
  contents: write
  pull-requests: write
  id-token: write

jobs:
  review:
    if: |
      github.event_name == 'workflow_dispatch' ||
      github.event_name == 'pull_request' ||
      (github.event_name == 'deployment_status' && github.event.deployment_status.state == 'success')
    runs-on: ubuntu-latest
    timeout-minutes: 25
    steps:
      - uses: actions/checkout@v4
      - uses: marcushyett/tik-test@v1
        with:
          claude-code-oauth-token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
          openai-api-key:          ${{ secrets.OPENAI_API_KEY }}            # optional
          vercel-bypass-secret:    ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}  # optional
          pr-number:               ${{ github.event.inputs.pr_number }}     # workflow_dispatch only

The action installs Node + ffmpeg + Playwright, builds tik-test, auto-detects the PR number and preview URL, runs the review, posts back.

Inputs

Plus more typed inputs for fine-tuning (feature-finder-timeout, max-goals, min-chunk-seconds, max-body-scenes, checklist-min-items, checklist-max-items, intro-seconds, outro-seconds, outro-hold-seconds, render-segments, render-concurrency, video-cache-mb, node-max-old-space-mb). See Advanced or node dist/cli.js config.

¹ One of the two. OAuth recommended.

Secrets

For the full end-to-end walkthrough — every token, where to create it, and where to paste it (including the optional reviewer web app + self-review CI) — see docs/SETUP.md.

Gating

• review-mode: request-changes-on-fail (default) blocks merge on repos requiring approval.
• require-pass: true (default) turns the check red on failure.
• Set require-pass: false to keep the check green and rely only on the formal review.

Self-hosted reference

This repo dogfoods the action with two path-scoped workflows: tik-test-taskpad.yml (runs on PRs touching examples/todo-app/**) and tik-test-webapp.yml (runs on PRs touching web/**). Both use uses: ./ instead of marcushyett/tik-test@v1.

Troubleshooting

"Plan generation timed out after 240000ms"
PR diff is huge or tiktest.md is missing. Bump plan-timeout: 480 or trim the diff.

"Per-goal agent timed out after 600000ms"
Agent stuck. Page didn't load, login is broken, or the goal is too vague. Check the last mcp__playwright_* tool call. If your app is just slow, bump agent-timeout: 1200.

"Narration generation timed out after 540000ms"
Recording too long (15+ tool calls). Trim goals, bump narration-timeout: 900, or set TIK_MAX_BODY_SCENES=8.

Silent video on the GitHub runner
Set OPENAI_API_KEY. Without it, Linux runners ship silent.

Agent clicks the wrong button
Add a ## Hints section to claude.md describing the surface, or add data-testid attributes. The plan generator sees the diff but not the rendered DOM.

claude: command not found in CI
Use the GitHub Action; it bundles the install. Outside CI, run npm i -g @anthropic-ai/claude-code.

PR comment shows the marker but the video is broken
Expected fallback: if any post-process step crashes, tik-test still uploads the raw recording. Check run artifacts.

Error: Process completed with exit code 143 (kernel OOM kill)
Long captures + multiple parallel Chromium browsers can exhaust the standard private-repo runner's 7 GB envelope. The action ships safe defaults — render-segments: 1, video-cache-mb: 256, node-max-old-space-mb: 4096 — that should keep the job inside it. If you still see exit 143:

• Lower further: video-cache-mb: 128 (smallest practical), quick-and-dirty: true.
• Or upgrade the runner: runs-on: ubuntu-latest-8-cores (paid GitHub larger runner, ~32 GB / 8 vCPU). On a larger runner you can also speed the render up by raising render-segments: 3 and video-cache-mb: 512.
• A heap-OOM error from Node (instead of a silent SIGTERM) means it's V8's heap, not the kernel — bump node-max-old-space-mb: 6144.

Advanced

Run node dist/cli.js config to print every knob with its current value, default, override hint, and risks. Same data as below, scoped to your environment.

Defaults are tuned for a typical PR (1-3 goals, 30-60s recording, 8-12 narration scenes, 6-10 checklist items) on a Claude Max subscription. Bump only after you've seen the corresponding default fail in your run.

Every knob has a matching typed input on the GitHub Action (kebab-case version of the env-var, dropping the TIK_ prefix and _MS suffix). Prefer the typed input in YAML.

Voice / TTS

Auth

Debugging

Claude CLI timeouts (millisecond)

The Action exposes all four as typed seconds inputs (plan-timeout, agent-timeout, narration-timeout, feature-finder-timeout). Prefer those in YAML.

Body-narration density

Outro checklist sizing

Intro / outro durations

Contributing

See CONTRIBUTING.md.

# 1. Change code, 2. rebuild, 3. dogfood:
npm run build
OPENAI_API_KEY=... node dist/cli.js run --config examples/todo-app/tiktest.md --quick

License

MIT. See LICENSE.