Skillget

awesome-ai-video-editing

mcp
Security Audit
Warn
Health Warn
  • License — License: NOASSERTION
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Low visibility — Only 5 GitHub stars
Code Pass
  • Code scan — Scanned 1 files during light audit, no dangerous patterns found
Permissions Pass
  • Permissions — No dangerous permissions requested

No AI report is available for this listing yet.

SUMMARY

A curated list of AI video editing tools, agentic editors, MCP servers, open-source projects, models and papers.

README.md

Awesome AI Video Editing Awesome

A curated list of AI and agentic tools for editing video — plus the MCP servers, open-source
projects, models, papers and datasets underneath them.

Maintained by Valmera, a commercial editor included in this list.
See the contribution and affiliation policy.

Verification history: the original directory was checked on 2026-08-05, with newer
agentic-editor additions checked on 2026-10-06. On 2026-11-06, the workflow categories and
Valmera descriptions were revised. On 2026-11-07, the Grok Imagine entry was added from its
official API documentation and the generative-workflow description was corrected; this was
a documentation check, not a hands-on test. Other product descriptions, star counts, licences and
activity dates retain their earlier checks; use each project's current documentation before
choosing a tool.

Choose the workflow you need

AI video products can combine several workflows. These categories describe the task a reader
wants to perform, and a product may support more than one.

Workflow What happens Starting material
Agentic editing You describe an outcome; an agent plans and performs editing operations Footage, project context and a creative brief
AI-assisted editing A person directs the edit and uses AI for selected operations A timeline, source media or a selected task
Generative video A model creates or transforms visual material A prompt, reference media or both

For an editing project, identify the source files, the new material you need, the review process
and the final deliverable. Check the selected product's current tools, account requirements and
export options. A generated shot may become source material in a larger edit; an editor may
include generation within that workflow.

Compare an edit, a revision and the delivered file

A feature list helps build a shortlist. To compare results, use the same recording and request
one correction after the first preview. Our interview-editing comparison sheet
provides copyable briefs, a pause-restoration exercise, caption checks and a blank results table.
It separates documented features, observed behavior and work that remains untested. It is an
evaluation method, not a ranking or a claim that every listed tool passes.

For a rough cut that needs one shorter shot, compare four revision workflows and trace the source frames. The exercise includes a downloadable timecoded recording and the expected positions in both cuts.

The sheet also links downloadable, vendor-authored examples
and explains what their measurements establish. A recorded caption-style revision and a
fictional interview paper edit are labeled separately; neither is a multi-product benchmark.

Contents

Agentic video editors

An agent takes a natural-language goal and performs the edit on footage you supply. This is a
young category — most entries are under two years old and every one of them has real limits,
noted here.

  • Cardboard - Browser editor where an agent edits on request and can also generate media; a Mac app lets your own Claude or Codex agent drive it. (hosted, Mac app)
  • ChatCut - Chat-based editor with built-in video models and XML handoff to traditional NLEs. (hosted, desktop)
  • Cutback - Assistant for Premiere Pro that pre-builds selects and rough cuts from raw and multi-cam footage. (hosted, Premiere plugin)
  • Diffusion Studio Agent - Framework where an LLM writes and runs browser-based compositing code to fulfil an editing request. (MIT · 275★ · last commit 2025-02)
  • invideo AI agent - An AI agent on a full multitrack editor, with generation, team features and remote agents over MCP. (hosted)
  • Loopdesk - Builds an AI first cut across a library of creator footage. (hosted)
  • Mosaic - Node canvas where agents run edits on autopilot and produce A/B variants from one set of rushes, with a timeline to take over. (hosted)
  • Riverside Co-Creator - Agent inside Riverside for podcasts and interviews recorded on the platform, with an MCP server. (hosted, requires Riverside)
  • Underlord (Descript) - Agent inside Descript that acts on editing instructions across a transcript-backed timeline. (hosted, requires Descript)
  • Valmera - Edits uploaded footage from a conversational brief, with versioned changes, rendered previews and final video export. (hosted · free account, paid editing)
  • VideoAgent (HKUDS) - Research framework combining video understanding, editing and remaking behind one agent loop. (MIT · 1.6k★)
  • Vyra - Chat editing on a full timeline, driven by its built-in AI or your own assistant over MCP; the project stays open in a browser tab while it edits. (hosted)
  • agentic-video-editor - Turns raw footage plus a creative brief into an ad using an ensemble of Gemini agents over FFmpeg. (MIT · 467★)

AI-assisted editors

A human drives the timeline; AI removes the chores. Includes the plugins that add AI to a
conventional NLE.

  • Adobe Premiere - Industry-standard NLE; ships Text-Based Editing, Generative Extend, Enhance Speech and Media Intelligence search. (paid)
  • AutoCut - Premiere Pro and DaVinci Resolve plugin for silence removal, captions, B-roll, zooms and podcast multi-cam. (paid plugin)
  • Camtasia - Screen recorder and editor aimed at tutorials and software demos, with AI captions and voice. (paid · free trial)
  • Canva Video Editor - Browser drag-and-drop editor with templates, brand kits and AI generation built into the same canvas. (hosted · free tier)
  • CapCut - Free social-first editor on web, desktop and mobile with auto-captions, reframing and effect templates. (hosted and desktop · free tier)
  • Captions - Talking-head editor that handles cuts, captions, eye-contact correction and dubbing from a phone or browser. (hosted)
  • Clipchamp - Microsoft's browser editor with auto-compose, speaker coach and text-to-speech, free with a Microsoft account. (hosted · free tier)
  • Clueso - Turns a rough screen recording into a narrated, on-brand product video and a written step-by-step guide. (hosted)
  • DaVinci Resolve - Editing, colour, VFX and audio post in one app; the Neural Engine does tracking, isolation, reframing and speech cleanup. (desktop · free tier)
  • Descript - Edits video by editing its transcript; removes filler words, clones voice and fixes gaze. (hosted · free tier)
  • Filmora - Consumer desktop and mobile editor with AI masking, smart cutout and motion tracking. (paid · free trial)
  • Final Cut Pro - Apple's Mac and iPad NLE, with magnetic timeline, scene-removal mask and voice isolation. (paid)
  • FireCut - Plugin that automates silence removal, captions, zoom cuts and chapters inside Premiere Pro and Resolve. (paid plugin · free tier)
  • Gling - Removes bad takes and silences from talking-head YouTube footage and hands back a timeline. (hosted)
  • Kapwing - Collaborative browser editor that can also assemble a whole project from a single prompt. (hosted · free tier)
  • Movavi Video Editor - Lightweight consumer editor for Windows and Mac with background removal and motion tracking. (paid · free trial)
  • Recut - Desktop app that strips silence from long recordings and exports to Premiere, Resolve or Final Cut. (paid)
  • Riverside - Records local-quality remote podcasts and video, then edits by transcript and cuts clips. (hosted · free tier)
  • Runway - Generation plus a working toolbox: inpainting, rotoscoping, motion tracking and green-screen removal. (hosted · free tier)
  • TimeBolt - Jump-cuts silence, filler words and bad takes out of talking videos and podcasts. (paid)
  • Topaz Video - Model-based upscaling, denoise, deinterlacing and frame interpolation for archival and low-light footage. (paid)
  • VEED - Browser editor combining subtitles, dubbing, avatars and a template library in one workflow. (hosted · free tier)
  • Wisecut - Auto-cuts long talking video into short clips, adds captions, ducks background music. (hosted · free tier)

Clipping and repurposing

Long video in, short vertical clips out.

  • 2short.ai - Turns long YouTube uploads into shorts, aimed at channel growth rather than social scheduling. (hosted)
  • AI-Youtube-Shorts-Generator - Open-source clipper using LLM highlight detection, Whisper and face-aware 9:16 reframing. (no licence file · 4.5k★)
  • Clips AI - Python library that converts long video into clips and reframes them; transcript-driven. (MIT · 524★ · last commit 2024-01)
  • Crayo - Short-form generator for gameplay-and-voiceover formats with subtitles and stock backgrounds. (hosted)
  • Klap - One-click clipping of podcasts and webinars with captions, reframing and a virality score. (hosted)
  • MoneyPrinterTurbo - Generates full short videos from a topic or keyword through an automated LLM plus stock pipeline. (MIT · 102k★)
  • OpusClip - The most widely cited clipper; scores moments, reframes to vertical and adds animated captions. (hosted · free tier)
  • Podsqueeze - Podcast-focused: transcripts, summaries, show notes and audiogram clips from one upload. (hosted · free tier)
  • Reap - Clips, captions and dubs long video into 100+ languages, with an API for pipeline use. (hosted)
  • ShortGPT - Experimental framework for automating short-form channels end to end. (MIT · 7.8k★ · last commit 2025-02)
  • Spikes Studio - Clip generator built around Twitch and livestream footage as well as YouTube. (hosted)
  • Submagic - Caption-led short-form editor with B-roll, zooms and sound effects applied automatically. (hosted)
  • Vizard - Long-to-short clipping with editable transcript, captions and a template library. (hosted · free tier)
  • quso.ai - Clips long video and schedules the results straight to social accounts. (hosted · free tier)

Captions and subtitles

Burn-in, translation, and the editors that handle subtitle files properly.

  • Aegisub - The reference typesetting subtitle editor for ASS/SSA; still what fansub-grade karaoke and effects are built in. (3.4k★ · last commit 2019-10; community forks are more active)
  • Buzz - Desktop app that transcribes and translates offline with Whisper, exporting SRT, VTT and more. (MIT · 20.8k★)
  • Happy Scribe - Transcription and subtitling in 150+ languages with human-review upgrades. (hosted · free tier)
  • Maestra - Subtitles, transcripts and multilingual voiceover for on-demand and live media in 125+ languages. (hosted)
  • Notta - Meeting and interview transcription with summaries and subtitle export. (hosted · free tier)
  • Otter - Live meeting transcription and notes; useful as a source of timed text rather than as an editor. (hosted · free tier)
  • Rev - Human and AI transcription and captioning with a long track record on accuracy-critical work. (hosted)
  • Sonix - Transcription and subtitling in 49+ languages with an in-browser aligned editor. (hosted)
  • Subs AI - Web UI, CLI and Python package for generating subtitles with Whisper and its variants. (GPL-3.0 · 1.7k★)
  • Subtitle Edit - The most complete open-source subtitle editor: sync, OCR, waveform, 300+ formats. (MIT · 13.7k★)
  • Vibe - Cross-platform desktop transcription app that runs entirely on your machine. (MIT · 7.0k★)
  • Zeemo - Auto-captioning with styled templates aimed at social video. (hosted · free tier)

Video MCP servers

Model Context Protocol servers that expose video and audio
tools to an assistant. Hosted services, local rendering servers and bridges to installed editors
have different setup requirements. Check the linked source for the actual endpoint, authentication,
media-upload path and preview/export behavior; an API alone does not establish a hosted MCP service.

The Creatomate, JSON2Video and Shotstack entries below were checked against their official MCP
documentation on 2026-11-07. This verifies documented connection routes, not hands-on editing
quality. Use the comparison sheet to record your own edit, revision and export.

  • Claude Code Video Toolkit - Bundle of skills and MCP servers for Remotion, Manim, screen recording, YouTube clipping and FFmpeg. (no licence file · 64★)
  • Creatomate MCP - Connects an assistant to one hosted project to browse, draft and refine templates and inspect renders. (hosted service)
  • DaVinci Resolve MCP - Drives Resolve Studio through its official scripting API: timeline, media pool, render, grade, Fusion, Fairlight. (MIT · 2.0k★)
  • ElevenLabs MCP - Official server for speech synthesis, dubbing and voice cloning — the audio half of a video pipeline. (MIT · 1.5k★)
  • JSON2Video MCP - Offers remote and local connections for rendering, job status, movie validation, saved templates and media upload. (hosted service · optional local client)
  • Kinocut - Guardrailed local editing server over FFmpeg with repurposing tools, a Python client and a CLI. (Apache-2.0 · 99★)
  • Manim MCP Server - Renders Manim animations from an assistant, for explainer and motion-graphics inserts. (MIT · 626★)
  • MCP Server Whisper - Exposes audio transcription to an assistant, with batch processing of local files. (MIT · 56★)
  • Remotion MCP App - The model writes React/Remotion compositions, the server compiles them and a live player renders the result. (41★)
  • Shotstack MCP - Provides a hosted connection for Studio previews, timeline renders, render status, share links and reusable templates. (hosted service)
  • Valmera MCP - Connects assistants to hosted editing tools for uploaded footage, previews and final exports over authenticated MCP. (MIT docs · hosted service)
  • Video Editor MCP - Compact FFmpeg server covering trim, concat, overlay and format conversion. (no licence file · 50★)
  • Video Jungle MCP - Client for Video Jungle: upload, search and edit video from an assistant. (no licence file · 284★ · last commit 2025-10)
  • ffmpeg-mcp (AmolDerickSoans) - Broad wrapper covering decode, encode, transcode, mux, demux, stream and filter. (MIT · 14★ · last commit 2025-03)
  • ffmpeg-mcp (dubnium0) - 40+ tools spanning processing, analysis and streaming. (no licence file · 19★)
  • ffmpeg-mcp-server - Small, focused server for pulling highlight segments out of long recordings. (MIT · 3★)
  • mcp-server-youtube-transcript - Fetches YouTube transcripts directly into the model's context. (MIT · 581★)
  • mcp-video-editor - FFmpeg plus Whisper, so cuts can be made against what was actually said. (no licence file · 5★)
  • mcp-youtube-transcript - Transcript retrieval with language selection and chunking for long videos. (MIT · 463★)
  • vfx-mcp - Effects-oriented server for compositing and filter chains. (MIT · 8★)
  • vibevideo-mcp - FFmpeg agent server that also backs a human-facing editor front end. (no licence file · 217★ · last commit 2025-06)
  • video-audio-mcp - FFmpeg server for conversion, trimming, overlays, transitions and audio processing. (MIT · 83★ · last commit 2025-05)

Directories worth checking for newer servers: the official MCP registry,
mcp.so, mcpservers.org and
punkpeye/awesome-mcp-servers (92k★).

Note: Remotion's own hosted MCP is deprecated; its docs now point to Remotion Agent Skills instead.

Open-source projects and frameworks

  • Diffusion Studio Core - Browser compositing engine on WebCodecs; renders timelines client-side. (MPL-2.0 · 1.2k★)
  • FFCreator - Node.js library for building videos from images, text and transitions at speed. (MIT · 3.2k★ · last commit 2024-12)
  • Flowblade - Multitrack Linux NLE with a film-style, film-first workflow. (GPL-3.0 · 3.1k★)
  • HandBrake - The default answer for transcoding and batch conversion before or after an edit. (GPL-2.0 · 23.9k★)
  • Kdenlive - KDE's mature multitrack editor built on MLT; proxy editing, effects, keyframes. (GPL-3.0 · 5.4k★)
  • LosslessCut - Trims, cuts and merges without re-encoding — the fastest way to top and tail huge files. (GPL-2.0 · 42.6k★)
  • Motion Canvas - TypeScript library and editor for procedurally animated explainer video. (MIT · 18.9k★)
  • Olive - Node-based free NLE with a professional feature target. (GPL-3.0 · 9.1k★ · last commit 2024-12)
  • OpenShot - Cross-platform desktop editor with keyframes, titles and 3D animation, aimed at beginners. (GPL-3.0 · 6.1k★)
  • Pitivi - GTK non-linear editor built on GStreamer Editing Services. (GitHub mirror · 243★)
  • Remotion - Make videos programmatically in React; the standard for data-driven and templated output. (source-available, company licence required · 55.5k★)
  • Revideo - Fork of Motion Canvas turned into a programmatic video API for parameterised rendering. (MIT · 4.0k★)
  • Shotcut - Cross-platform editor on MLT with wide format support and a very active release cadence. (GPL-3.0 · 14.8k★)
  • auto-editor - Cuts silence and dead air from the command line, exporting to Premiere, Resolve and Final Cut. (Unlicense · 4.7k★)
  • autocut - Edit video by editing its transcript in a text editor; delete a line, lose the take. (Apache-2.0 · 7.8k★ · last commit 2024-10)
  • editly - Declarative CLI and Node API that renders an edit from a JSON spec. (MIT · 5.5k★ · last commit 2025-02)

Generative video

These tools create or transform visual material. Some accept existing video and regenerate
parts of a shot; others start from text, images or a script. Check source limits and preservation
requirements for the specific model. Generated or transformed shots can become inserts and
B-roll in a larger editing project.

Note: OpenAI's Sora app and website were discontinued on 2026-04-26 and its API retires
2026-09-24, so it is not listed despite still appearing in most roundups.

Hosted models and platforms

  • D-ID - Talking-avatar video from a photo and a script, with an interactive-agent product on top. (hosted)
  • Fliki - Turns scripts and blog posts into narrated video with 2,000+ voices in 80+ languages. (hosted · free tier)
  • Gemini Omni - Google's conversational video model; generates from text, image or video and re-generates edits like backgrounds and lighting on uploads. (hosted)
  • Grok Imagine - Transforms source video from text instructions; the editing API accepts clips up to 8.7 seconds and caps output at 720p. (hosted API · docs checked 2026-11-07)
  • Hailuo - MiniMax's text- and image-to-video model, known for character motion. (hosted · free tier)
  • Hedra - Character and performance video generation from image plus audio. (hosted)
  • HeyGen - Avatar video and translation with lip-sync across languages. (hosted)
  • Higgsfield - Camera-motion-led generation suite with an agent for repeat creative workflows. (hosted)
  • Kling - Kuaishou's video model, strong on longer shots and physical motion. (hosted · free tier)
  • Krea - Real-time creative suite for image, video and 3D with enhancement and editing passes. (hosted · free tier)
  • LTX Studio - Storyboard-to-video production environment with shot-level control, built on LTX models. (hosted)
  • Luma Dream Machine - Text and image to video with keyframe interpolation between stills. (hosted · free tier)
  • Midjourney - Image model with video generation; the reference for stylised look development. (hosted)
  • Pictory - Script and article to video using stock footage and synthetic narration. (hosted)
  • Pika - Short generative clips with an effects-led, consumer-facing interface. (hosted · free tier)
  • Runway - Gen-series video models alongside the editing toolbox listed above. (hosted · free tier)
  • Renderforest - Template-driven video, logo and website generation for marketing output. (hosted · free tier)
  • Synthesia - Enterprise avatar video from a script, with brand templates and localisation. (hosted)
  • Veo - Google DeepMind's video generation model, with native audio. (hosted)
  • Vidu - Text, image and reference-driven generation with consistent characters across shots. (hosted · free tier)
  • Wan - Alibaba's hosted front end for the open Wan models. (hosted · free tier)
  • invideo AI - Prompt-to-video agent that scripts, sources stock and narrates a full edit. (hosted · free tier)

Open models and tooling

  • Allegro - Text-to-video model producing 6-second 720p clips at 15fps. (Apache-2.0 · 1.1k★)
  • AnimateDiff - Motion module that animates existing Stable Diffusion checkpoints. (Apache-2.0 · 12.2k★ · last commit 2024-07)
  • CogVideo / CogVideoX - Zhipu's text- and image-to-video family with released weights and finetuning code. (Apache-2.0 · 12.9k★)
  • ComfyUI - Node-graph backend and GUI that most open video models are actually run through. (GPL-3.0 · 124k★)
  • Deforum - Keyframed camera-motion animation from diffusion models; the origin of the look. (2.3k★ · last commit 2024-08)
  • EasyAnimate - End-to-end transformer-diffusion pipeline for long, high-resolution generation. (Apache-2.0 · 2.3k★ · last commit 2025-03)
  • FramePack - Next-frame prediction that makes long video diffusion practical on modest GPUs. (Apache-2.0 · 17.2k★)
  • HunyuanVideo - Tencent's large open video generation framework with weights and inference code. (community licence · 12.4k★)
  • LTX-Video - Lightricks' fast DiT video model, tuned for real-time-ish local generation. (Apache-2.0 · 10.8k★)
  • Mochi 1 - Genmo's open video generation model with an emphasis on motion fidelity. (Apache-2.0 · 3.7k★)
  • Open-Sora - Full open reproduction of a Sora-style pipeline, including training code. (Apache-2.0 · 29.2k★)
  • Open-Sora-Plan - Parallel community reproduction effort with released checkpoints. (MIT · 12.2k★)
  • Stable Video Diffusion - Stability's image-to-video model and reference implementation. (MIT · 27.2k★)
  • VideoCrafter - Research video diffusion models for text-to-video and image-to-video. (5.1k★)
  • Wan 2.1 - Alibaba's open video model, widely used as a finetuning base. (Apache-2.0 · 16.7k★)
  • Wan 2.2 - The follow-up release, with a mixture-of-experts architecture. (Apache-2.0 · 17.0k★)

Speech, transcription and audio

Word-level timing is what makes transcript-driven and agentic editing possible at all.

  • AssemblyAI - Speech-to-text API with diarization, chapters and audio intelligence models. (hosted API · free tier)
  • Deepgram - Low-latency streaming and batch speech-to-text with a strong real-time story. (hosted API · free tier)
  • Demucs - Hybrid source separation; splits dialogue from music and effects. (MIT · 3.0k★)
  • ElevenLabs - Voice synthesis, cloning and dubbing used for voiceover and localisation. (hosted · free tier)
  • Montreal Forced Aligner - Kaldi-based forced alignment; the standard when phoneme-level timing matters. (MIT · 1.9k★)
  • NVIDIA NeMo - Production speech toolkit with Parakeet and Canary ASR models and diarization. (Apache-2.0 · 17.9k★)
  • Silero VAD - Fast, tiny voice-activity detector — the practical way to find silences to cut. (MIT · 9.9k★)
  • SpeechBrain - PyTorch toolkit spanning ASR, separation, enhancement and speaker tasks. (Apache-2.0 · 11.7k★)
  • Vosk - Offline speech recognition for 20+ languages on desktop, mobile and Raspberry Pi. (Apache-2.0 · 15.0k★)
  • Whisper - OpenAI's multilingual ASR model; the base almost everything below builds on. (MIT · 107k★)
  • WhisperX - Whisper with forced-alignment word timestamps and speaker diarization. (BSD-2-Clause · 23.4k★)
  • aeneas - Synchronises audio to an existing text without recognising it. (AGPL-3.0 · 2.9k★ · last commit 2020-05)
  • ctc-forced-aligner - Lightweight multilingual CTC forced alignment with no Kaldi dependency. (no licence file · 542★)
  • faster-whisper - CTranslate2 reimplementation, several times faster than the reference at equal accuracy. (MIT · 24.7k★)
  • ffmpeg-normalize - Batch EBU R128 and peak normalisation wrapper around FFmpeg. (1.5k★)
  • gentle - Robust forced aligner that returns word timings and flags the words it could not place. (MIT · 1.7k★)
  • insanely-fast-whisper - Batched, optimised Whisper CLI for bulk transcription. (Apache-2.0 · 13.0k★ · last commit 2024-05)
  • pyannote.audio - Speaker diarization toolkit — who spoke when, which is what multi-cam cutting needs. (MIT · 10.4k★)
  • pyloudnorm - ITU-R BS.1770-4 loudness measurement in Python, for hitting delivery LUFS targets. (MIT · 779★)
  • python-audio-separator - CLI and library wrapping many UVR stem-separation models. (MIT · 1.3k★)
  • stable-ts - Stabilises Whisper timestamps and adds alignment and audio indexing. (MIT · 2.3k★)
  • whisper-timestamped - Word-level timestamps and per-word confidence from Whisper directly. (AGPL-3.0 · 2.8k★)
  • whisper.cpp - C/C++ port that runs Whisper on CPU, phones and edge devices. (MIT · 52.6k★)

Libraries and building blocks

  • FFmpeg - The layer under nearly every tool on this page. (GPL-2.0 / LGPL · 63k★)
  • FFmpeg Filters Documentation - The filtergraph reference; the single most useful page for anyone building an editor.
  • MLT Framework - Multimedia authoring framework behind Shotcut and Kdenlive; models tracks and transitions. (LGPL-2.1 · 1.8k★)
  • Mediabunny - Pure TypeScript toolkit to read, write and convert media in the browser. (MPL-2.0 · 6.9k★)
  • MoviePy - The usual first library for scripted cutting, concatenation and compositing in Python. (MIT · 14.8k★)
  • OpenCV - Frame-level computer vision: tracking, face detection, cropping and reframing. (Apache-2.0 · 90k★)
  • PyAV - Pythonic bindings to FFmpeg's libraries for frame-accurate decode and encode. (BSD-3-Clause · 3.3k★)
  • PySceneDetect - Shot-boundary and cut detection; the standard way to segment footage before analysis. (BSD-3-Clause · 5.1k★)
  • TransNet V2 - Neural shot-transition detection, more accurate than threshold methods on hard cuts and dissolves. (MIT · 1.0k★ · last commit 2021-07)
  • aubio - Onset, pitch and beat detection in C with Python bindings. (3.7k★)
  • essentia - C++ audio analysis library covering rhythm, key and descriptors, with Python bindings. (AGPL-3.0 · 3.7k★)
  • ffmpeg-python - Filtergraph bindings for Python; widely used but unmaintained since 2022. (Apache-2.0 · 11.0k★ · last commit 2022-07)
  • ffmpeg.wasm - FFmpeg compiled to WebAssembly for in-browser processing with no upload. (MIT · 17.7k★)
  • librosa - Music and audio analysis in Python; beat tracking for cut-to-music workflows. (ISC · 8.5k★)
  • madmom - Signal-processing library with strong beat and downbeat trackers. (1.7k★ · last commit 2024-08)
  • yt-dlp - Downloader used to fetch source material and reference footage. (Unlicense · 182k★)

Research papers and benchmarks

Datasets

  • AMI Meeting Corpus - 100 hours of multi-modal meeting recordings; the classic diarization and multi-camera testbed.
  • ActivityNet Captions - Dense temporal event descriptions over 20k untrimmed videos.
  • AudioSet - 2M+ labelled 10-second audio clips across 632 event classes.
  • Common Voice - Mozilla's crowd-sourced multilingual speech corpus, CC0.
  • Condensed Movies - Key scenes from 3k+ films with captions, for story-level retrieval.
  • Ego4D - 3,600+ hours of egocentric video with dense annotations.
  • GigaSpeech - 10k hours of transcribed English audio from audiobooks, podcasts and YouTube.
  • HowTo100M - 136M narrated instructional clips paired with ASR text.
  • InternVid - Large-scale video-text corpus for multimodal understanding and generation.
  • LibriSpeech - 1,000 hours of read English speech; still the default ASR baseline.
  • MovieNet - 1,100 movies annotated with shots, scenes, characters, places and scripts.
  • Panda-70M - 70M high-quality video clips with captions from cross-modality teachers.
  • The Anatomy of Video Editing (AVE) - 196k shots labelled with editing attributes — the closest thing to a dataset of editing decisions.
  • VGGSound - 200k+ audio-visual clips where the sound source is visible on screen.
  • VidChapters-7M - 817k videos with user-written chapters and timestamps.
  • VoxConverse - Audio-visual diarization dataset from real multi-speaker broadcast.
  • YouCook2 - 2,000 cooking videos with temporally localised procedure descriptions.

Reading

Contributing

Additions and corrections are welcome — see CONTRIBUTING.md for the inclusion
bar and the conflict-of-interest disclosure. Links are checked automatically every week.

License

CC0

To the extent possible under law, the contributors have waived all copyright and related or
neighboring rights to this work.

Reviews (0)

No results found