ffmpeg-media

FFmpeg Media Processing Skill

Dedicated to sudokku, creator of micro-ffmpeg — the in-browser video editor that proved ffmpeg.wasm is production-ready. Built in 2 days, 2,489 LOC, 93 commits. The patterns in this skill are drawn from that work.

Expert-level ffmpeg command construction, media pipeline design, and ffmpeg.wasm integration patterns. Covers CLI ffmpeg, Node.js ffmpeg bindings, and browser-side ffmpeg.wasm.

Core Principle

Build the filter graph first, then wrap it in the ffmpeg command. Never guess at flags — construct from codec specs.

Quick Reference

Probe a File

# Full probe (JSON output)
ffprobe -v quiet -print_format json -show_format -show_streams input.mp4

# Quick summary
ffprobe -v error -show_entries format=duration,bit_rate -show_entries stream=codec_name,width,height,r_frame_rate input.mp4

Common Conversions

# MP4 (H.264 + AAC) — universal compatibility
ffmpeg -i input.mkv -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k output.mp4

# WebM (VP9 + Opus) — web-optimized, smaller files
ffmpeg -i input.mp4 -c:v libvpx-vp9 -crf 30 -b:v 0 -c:a libopus -b:a 128k output.webm

# H.265/HEVC — 50% smaller than H.264 at same quality
ffmpeg -i input.mp4 -c:v libx265 -preset medium -crf 28 -c:a aac -b:a 128k output.mp4

# AV1 — best compression, slow encoding
ffmpeg -i input.mp4 -c:v libaom-av1 -crf 30 -b:v 0 -c:a libopus output.mkv

# Audio only (MP3)
ffmpeg -i input.mp4 -vn -c:a libmp3lame -q:a 2 output.mp3

# Audio only (high quality FLAC)
ffmpeg -i input.wav -c:a flac output.flac

Trim / Cut

# Trim from 00:01:30 for 60 seconds (fast, no re-encode)
ffmpeg -ss 00:01:30 -i input.mp4 -t 60 -c copy output.mp4

# Trim with re-encode (frame-accurate)
ffmpeg -i input.mp4 -ss 00:01:30 -to 00:02:30 -c:v libx264 -c:a aac output.mp4

Scale / Resize

# Scale to 1280x720 (maintain aspect ratio)
ffmpeg -i input.mp4 -vf "scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1" output.mp4

# Scale to 50%
ffmpeg -i input.mp4 -vf "scale=iw/2:ih/2" output.mp4

# Scale for social media (vertical 9:16)
ffmpeg -i input.mp4 -vf "scale=1080:1920:force_original_aspect_ratio=increase,crop=1080:1920" output.mp4

Filters

# Blur
ffmpeg -i input.mp4 -vf "boxblur=5:1" output.mp4

# Brightness/Contrast/Saturation
ffmpeg -i input.mp4 -vf "eq=brightness=0.1:contrast=1.2:saturation=1.3" output.mp4

# Speed up 2x (video + audio)
ffmpeg -i input.mp4 -vf "setpts=0.5*PTS" -af "atempo=2.0" output.mp4

# Slow down 0.5x
ffmpeg -i input.mp4 -vf "setpts=2.0*PTS" -af "atempo=0.5" output.mp4

# Rotate 90 degrees clockwise
ffmpeg -i input.mp4 -vf "transpose=1" output.mp4

# Crop (center crop to 640x480)
ffmpeg -i input.mp4 -vf "crop=640:480" output.mp4

# Overlay watermark
ffmpeg -i input.mp4 -i watermark.png -filter_complex "overlay=W-w-10:H-h-10" output.mp4

# Text overlay
ffmpeg -i input.mp4 -vf "drawtext=text='Title':fontsize=48:fontcolor=white:x=(w-text_w)/2:y=50" output.mp4

Concatenation

# Create file list
echo "file 'clip1.mp4'" > list.txt
echo "file 'clip2.mp4'" >> list.txt
echo "file 'clip3.mp4'" >> list.txt

# Concat (same codec — fast)
ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

# Concat (different codecs — re-encode)
ffmpeg -f concat -safe 0 -i list.txt -c:v libx264 -c:a aac output.mp4

Thumbnails / Stills

# Single frame at 5 seconds
ffmpeg -i input.mp4 -ss 00:00:05 -frames:v 1 thumb.jpg

# Thumbnail grid (4x4)
ffmpeg -i input.mp4 -vf "select=not(mod(n\,300)),scale=320:180,tile=4x4" -frames:v 1 grid.jpg

# Extract frame every N seconds
ffmpeg -i input.mp4 -vf "fps=1/10" frames/frame_%04d.jpg

Audio Waveform Visualization

# PNG waveform
ffmpeg -i input.mp3 -filter_complex "showwavespic=s=1920x200:colors=white" -frames:v 1 waveform.png

# Video waveform
ffmpeg -i input.mp3 -filter_complex "showwaves=s=1280x720:mode=line:colors=white" waveform.mp4

Streaming (HLS/DASH)

# HLS output (adaptive bitrate)
ffmpeg -i input.mp4 -c:v libx264 -c:a aac \
  -hls_time 6 -hls_playlist_type vod \
  -hls_segment_filename "segment_%03d.ts" playlist.m3u8

# Multiple quality HLS
ffmpeg -i input.mp4 \
  -map 0:v -map 0:a -map 0:v -map 0:a \
  -c:v libx264 -c:a aac \
  -b:v:0 5M -s:v:0 1920x1080 \
  -b:v:1 1M -s:v:1 640x360 \
  -var_stream_map "v:0,a:0 v:1,a:1" \
  -hls_time 6 -hls_playlist_type vod \
  -master_pl_name master.m3u8 \
  -hls_segment_filename "v%v/seg_%03d.ts" "v%v/playlist.m3u8"

Subtitles

# Burn subtitles into video
ffmpeg -i input.mp4 -vf "subtitles=subs.srt" output.mp4

# Extract subtitles
ffmpeg -i input.mkv -map 0:s:0 subs.srt

ffmpeg.wasm Integration (Browser)

Setup (from micro-ffmpeg pattern)

// ffmpegSingleton.ts — main-thread singleton (no Comlink)
import { FFmpeg } from '@ffmpeg/ffmpeg';
import { fetchFile, toBlobURL } from '@ffmpeg/util';

let ffmpeg: FFmpeg | null = null;

export async function getFFmpeg(): Promise<FFmpeg> {
  if (ffmpeg) return ffmpeg;
  ffmpeg = new FFmpeg();
  
  await ffmpeg.load({
    coreURL: await toBlobURL('/ffmpeg-core.js', 'text/javascript'),
    wasmURL: await toBlobURL('/ffmpeg-core.wasm', 'application/wasm'),
  });
  
  return ffmpeg;
}

// Usage
const ff = await getFFmpeg();
await ff.writeFile('input.mp4', await fetchFile(file));
await ff.exec(['-i', 'input.mp4', '-vf', 'scale=640:360', 'output.mp4']);
const data = await ff.readFile('output.mp4');

Progress Tracking

ff.on('progress', ({ progress, time }) => {
  console.log(`${Math.round(progress * 100)}% (${time}μs)`);
});

Extract Thumbnails (Browser)

async function extractThumbnail(file: File, timeSeconds: number): Promise<Blob> {
  const ff = await getFFmpeg();
  await ff.writeFile('input', await fetchFile(file));
  await ff.exec([
    '-i', 'input',
    '-ss', String(timeSeconds),
    '-frames:v', '1',
    '-f', 'image2',
    'thumb.jpg'
  ]);
  const data = await ff.readFile('thumb.jpg');
  return new Blob([data], { type: 'image/jpeg' });
}

Codec Reference

| Codec | Use Case | Quality/Size | Speed | |-------|----------|-------------|-------| | H.264 (libx264) | Universal playback | Good/Medium | Fast | | H.265 (libx265) | Storage, streaming | Better/Small | Slow | | VP9 (libvpx-vp9) | Web delivery | Better/Small | Slow | | AV1 (libaom-av1) | Archive, future web | Best/Smallest | Very slow | | ProRes (prores_ks) | Editing intermediate | Lossless/Huge | Fast | | AAC | Universal audio | Good | Fast | | Opus | Web audio, VoIP | Better at low bitrates | Fast | | FLAC | Lossless audio | Lossless/Large | Fast |

CRF Quality Guide (H.264)

| CRF | Quality | Use Case | |-----|---------|----------| | 0 | Lossless | Archival | | 18 | Visually lossless | High-quality master | | 23 | Default | Good balance | | 28 | Lower quality | Web/mobile delivery | | 35+ | Very compressed | Thumbnails, previews |

Common Mistakes

| Mistake | Fix | |---------|-----| | -ss after -i (slow seek) | Put -ss before -i for fast seek | | Missing -movflags +faststart | Add for web-playable MP4 | | Using -b:v with CRF | CRF and bitrate are mutually exclusive | | Forgetting -pix_fmt yuv420p | Required for broad compatibility | | Concat without same codecs | Re-encode or use filter_complex | | Not setting thread count | Add -threads 0 for auto |

Gource Pipeline Integration

The gource-visualizer skill (skills/gource-visualizer/) produces PPM frame streams that pipe through ffmpeg. Key patterns:

# Standard Gource → ffmpeg pipeline (headless with xvfb)
xvfb-run -a gource --output-ppm-stream - \
  --viewport 1920x1080 -s 0.5 --auto-skip-seconds 1 \
  --stop-at-end /path/to/repo | \
ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i - \
  -vcodec libx264 -preset medium -crf 18 \
  -pix_fmt yuv420p -movflags +faststart output.mp4

# Cinema