claude-video-audio

claude-video-audio — Audio Processing

Pre-Flight

1. Run bash scripts/preflight.sh "$INPUT" "$OUTPUT"
2. Analyze audio streams: ffprobe -v error -select_streams a -show_entries stream=codec_name,channels,sample_rate,bit_rate -of json "$INPUT"

Loudness Normalization (EBU R128)

Two-pass workflow for accurate normalization:

# Pass 1: Measure current loudness
ffmpeg -i "$INPUT" -af loudnorm=I=-14:TP=-1.5:LRA=11:print_format=json -f null - 2>&1 | tail -12

Extract measured_I, measured_TP, measured_LRA, measured_thresh, offset from the JSON output.

# Pass 2: Apply normalization with measured values
ffmpeg -n -i "$INPUT" -af "loudnorm=I=-14:TP=-1.5:LRA=11:\
measured_I=MEASURED_I:measured_TP=MEASURED_TP:measured_LRA=MEASURED_LRA:\
measured_thresh=MEASURED_THRESH:offset=MEASURED_OFFSET:linear=true" \
  -c:v copy "$OUTPUT"

Target Loudness by Platform

| Platform | Target LUFS | True Peak | Notes | |----------|------------|-----------|-------| | YouTube | -14 LUFS | -1.0 dBTP | YouTube normalizes down, not up | | Spotify | -14 LUFS | -1.0 dBTP | "Loud" mode = -11 LUFS | | Apple Music | -16 LUFS | -1.0 dBTP | Sound Check | | Podcast | -16 LUFS | -1.5 dBTP | Conversational clarity | | Broadcast (EBU R128) | -23 LUFS | -1.0 dBTP | European standard | | TikTok/Reels | -14 LUFS | -1.0 dBTP | Match YouTube |

Noise Reduction

FFT-based (good for consistent background noise):

ffmpeg -n -i "$INPUT" -af "afftdn=nf=-25:nt=w" -c:v copy "$OUTPUT"

nf = noise floor in dB (lower = more aggressive). nt=w = white noise model.

Non-local means (better quality, slower):

ffmpeg -n -i "$INPUT" -af "anlmdn=s=7:p=0.002:r=0.002" -c:v copy "$OUTPUT"

Highpass/lowpass to remove rumble and hiss:

ffmpeg -n -i "$INPUT" -af "highpass=f=80,lowpass=f=12000" -c:v copy "$OUTPUT"

Silence Detection and Removal

Detect silence:

ffmpeg -i "$INPUT" -af "silencedetect=noise=-30dB:d=0.5" -f null - 2>&1 | grep silence

Returns silence_start and silence_end timestamps.

Remove silence (using Auto-Editor — much easier):

auto-editor "$INPUT" --margin 0.3s -o "$OUTPUT"

Manual silence removal with FFmpeg:

ffmpeg -n -i "$INPUT" -af "silenceremove=start_periods=1:start_silence=0.5:start_threshold=-30dB:detection=rms" -c:v copy "$OUTPUT"

Audio Extraction

# Extract as WAV (lossless)
ffmpeg -n -i "$INPUT" -vn -c:a pcm_s16le audio.wav

# Extract as AAC
ffmpeg -n -i "$INPUT" -vn -c:a aac -b:a 256k audio.m4a

# Extract as MP3
ffmpeg -n -i "$INPUT" -vn -c:a libmp3lame -b:a 320k audio.mp3

# Extract as Opus (most efficient)
ffmpeg -n -i "$INPUT" -vn -c:a libopus -b:a 128k audio.ogg

# Stream copy (fastest, keeps original codec)
ffmpeg -n -i "$INPUT" -vn -c:a copy audio.m4a

Audio Replacement

# Replace audio track entirely
ffmpeg -n -i video.mp4 -i new_audio.wav -c:v copy -c:a aac -b:a 192k \
  -map 0:v:0 -map 1:a:0 "$OUTPUT"

# Mix original audio with new audio (e.g., background music)
ffmpeg -n -i video.mp4 -i music.mp3 -filter_complex \
  "[0:a]volume=1.0[orig];[1:a]volume=0.3[music];[orig][music]amix=inputs=2:duration=first" \
  -c:v copy "$OUTPUT"

Audio Effects

Volume adjustment:

ffmpeg -n -i "$INPUT" -af "volume=1.5" -c:v copy "$OUTPUT"   # 1.5x louder
ffmpeg -n -i "$INPUT" -af "volume=-6dB" -c:v copy "$OUTPUT"   # 6dB quieter

Equalization:

# Boost bass, cut muddy mids, add presence
ffmpeg -n -i "$INPUT" -af "equalizer=f=80:t=q:w=1:g=3,equalizer=f=300:t=q:w=2:g=-2,equalizer=f=4000:t=q:w=1:g=2" -c:v copy "$OUTPUT"

Compression (reduce dynamic range):

ffmpeg -n -i "$INPUT" -af "acompressor=threshold=-20dB:ratio=4:attack=5:release=50" -c:v copy "$OUTPUT"

Limiter (prevent clipping):

ffmpeg -n -i "$INPUT" -af "alimiter=limit=0.95:level=false" -c:v copy "$OUTPUT"

Fade in/out:

ffmpeg -n -i "$INPUT" -af "afade=t=in:st=0:d=2,afade=t=out:st=OFFSET:d=2" -c:v copy "$OUTPUT"

Bass boost: -af "bass=g=5:f=100:w=0.5" Treble boost: -af "treble=g=3:f=4000:w=0.5"

Audio Format Conversion

| Codec | FFmpeg Encoder | Recommended Bitrate | Use Case | |-------|---------------|-------------------|----------| | AAC | aac | 128-256k | Video, streaming | | MP3 | libmp3lame | 192-320k | Universal playback | | Opus | libopus | 96-160k | Best efficiency, WebM | | FLAC | flac | Lossless | Archival, editing | | WAV | pcm_s16le | Uncompressed | Editing, processing | | AC3 | ac3 | 384-640k | Surround sound |

Channel Layout

# Stereo to mono
ffmpeg -n -i "$INPUT" -ac 1 -c:v copy "$OUTPUT"

# Mono to stereo (duplicate)
ffmpeg -n -i "$INPUT" -ac 2 -c:v copy "$OUTPUT"

# Extract left channel only
ffmpeg -n -i "$INPUT" -af "pan=mono|c0=FL" -c:v copy "$OUTPUT"

Audio Sync Fix

# Delay audio by 0.5 seconds
ffmpeg -n -i "$INPUT" -itsoffset 0.5 -i "$INPUT" -map 0:v -map 1:a -c copy "$OUTPUT"

# Advance audio by 0.5 seconds
ffmpeg -n -i "$INPUT" -itsoffset -0.5 -i "$INPUT" -map 0:v -map 1:a -c copy "$OUTPUT"

Audio Ducking (Sidechain Compression)

Automatically lower music volume when speech is present.

Basic ducking (voice track ducks music track):

ffmpeg -n -i video_with_voice.mp4 -i music.mp3 -filter_complex \
  "[0:a]aformat=fltp:44100:stereo[voice];\
   [1:a]aformat=fltp:44100:stereo[music];\
   [music][voice]sidechaincompress=threshold=0.015:ratio=6:attack=200:release=1000:level_sc=1[ducked];\
   [voice][ducked]amix=inputs=2:duration=first:weights=1 0.4[out]" \
  -map 0:v -map "[out]" -c:v copy "$OUTPUT"

Parameters guide: | Parameter | Default | Purpose | |-----------|---------|---------| | threshold | 0.015 | Voice level that triggers ducking (lower = more sensitive) | | ratio | 6 | How much to reduce music (higher = more reduction) | | attack | 200ms | How quickly music ducks down | | release | 1000ms | How quickly music returns after speech stops | | weights | 1 0.4 | Voice volume : music volume in final mix |

Ducking with existing audio streams (video has voice + separate music file):

ffmpeg -n -i "$INPUT" -i music.mp3 -filter_complex \
  "[1:a][0:a]sidechaincompress=threshold=0.02:ratio=8:attack=100:release=800[ducked];\
   [0:a][ducked]amix=inputs=2:duration=first:weights=1 0.3[out]" \
  -map 0:v -map "[out]" -c:v copy "$OUTPUT"

Tips:

• Lower threshold (e.g., 0.01) for quieter speech
• Higher ratio (e.g., 10-20) for more aggressive ducking
• Shorter attack (e.g., 50ms) for spoken word podcasts
• Longer release (e.g., 2000ms) for smoother music return
• Adjust weights to control the overall voice-to-music balance

Reference

Load references/audio.md for extended EBU R128 loudness details, advanced filter chains, and format-specific audio settings.