video-clipper

Video Clipper

Takes a long-form video and produces ready-to-post short-form vertical clips with speaker-tracked framing and professional animated captions. Works with podcasts, interviews, talks, and any talking-head content.

Requirements

• FFmpeg installed and available in PATH (brew install ffmpeg on macOS, apt install ffmpeg on Linux)
• Python 3 with openai-whisper and requests packages (pip install openai-whisper requests). Note: openai-whisper installs PyTorch (~2GB download). This skill uses openai-whisper instead of the lighter whisper-cpp because it provides word-level timestamps needed for accurate viral moment scoring.
• yt-dlp installed (for YouTube/URL downloads) — brew install yt-dlp on macOS, pip install yt-dlp on Linux
• API Keys in .env file (project root or any parent directory):
• KLAP_API_KEY — from klap.app (reframing with speaker tracking)
• CAPTIONS_AI_API_KEY — from captions.ai / platform.mirage.app (animated captions)

Before starting: Verify that FFmpeg, yt-dlp, and the Python packages are installed. If any are missing, instruct the user to install them before proceeding.

Cost Per Clip

| Step | Cost | |---|---| | Whisper (transcription) | Free (local) | | FFmpeg (clip extraction) | Free (local) | | Klap (reframing) | ~$1.50-2.50/clip depending on plan | | Captions.ai (captions) | ~$0.15/min of output | | Total per clip | ~$2-3 |

Input

The user provides:

1. Video source (required) — one of:
2. Local file path — e.g. /path/to/podcast.mp4
3. YouTube URL — e.g. https://www.youtube.com/watch?v=...
4. Any public video URL — direct link to MP4

1. Moment selection mode (ask the user):
2. Automatic — Claude picks the best moments
3. Manual — user provides specific timestamps
4. Hybrid — Claude proposes moments, user approves/adjusts before processing

1. Number of clips (optional) — default 3-5. Depends on video length and content density.

1. Caption template (optional) — Captions.ai template ID. Default: ctpl_DxflLOnuKkb198FNdI9E (Heat). List available templates via the API if user wants to browse.

1. Target clip duration (optional) — default 15-60 seconds. User can specify a range.

Pipeline

Step 1: Get the Video

Based on input type:

Local file:

# Verify it exists and get duration
ffprobe -v quiet -print_format json -show_format "video.mp4"

YouTube URL:

yt-dlp -f "bestvideo[height<=720]+bestaudio/best[height<=720]" --merge-output-format mp4 -o "<workdir>/source.mp4" "<URL>"

Other URL:

curl -L -o "<workdir>/source.mp4" "<URL>"

Step 2: Transcribe with Whisper

import whisper

model = whisper.load_model("base")
result = model.transcribe("source.mp4", language="en", word_timestamps=True)

Save both:

• transcript.json — full result with word-level timestamps (needed for Step 3)
• transcript.txt — readable version with timestamps per segment (for Claude to analyze)

Step 3: Identify Best Moments (Viral Scoring)

This is the key intelligence step. Claude reads the full transcript and identifies potential clip moments.

Step 3a: Segment the transcript into candidate moments

Scan the transcript for self-contained 15-60 second windows. Look for natural start/end points (topic changes, pauses, complete thoughts).

Step 3b: Score each candidate moment on this rubric

For each candidate, score 1-10 on these five criteria:

| Criteria | What to look for | Score guide | |---|---|---| | Hook Strength | Does the first sentence grab attention? Is it a surprising claim, provocative question, or bold statement? | 10 = "wait, what?" reaction. 1 = generic setup | | Quotability | Contains a memorable one-liner that people would screenshot or share? | 10 = tweet-worthy standalone quote. 1 = no standalone phrases | | Emotional Intensity | Does the speaker show passion, humor, anger, vulnerability, or conviction? | 10 = genuine emotion. 1 = monotone/flat delivery | | Self-Containedness | Does it make complete sense without watching the rest of the video? | 10 = fully standalone. 1 = needs prior context | | Surprise/Controversy | Does it challenge conventional wisdom, reveal something unexpected, or take a hot take? | 10 = counterintuitive insight. 1 = commonly known information |

Total score = sum of all five (max 50).

Step 3c: Rank and select top N moments

• Sort by total score descending
• Select top N (user-specified or default 3-5)
• Ensure selected moments don't overlap
• Prefer variety in topics/angles — don't pick 3 clips about the same point

Step 3d: Present to user for approval

For each selected moment, show:

• Timestamp range (start - end)
• Duration
• Transcript excerpt (first 2-3 lines)
• Score breakdown (hook/quotability/emotion/self-contained/surprise)
• Total score
• Suggested hook text for the clip

Wait for user approval. User can:

• Approve all
• Remove specific clips
• Add their own timestamps
• Adjust start/end times
• Request more options

Do NOT proceed to Step 4 until user approves.

Step 4: Extract Raw Clips

For each approved moment, extract with FFmpeg:

ffmpeg -y -ss <start> -to <end> -i source.mp4 -c copy clip<N>-raw.mp4

Step 5: Reframe with Klap

Upload each raw clip to Klap for AI-powered speaker-tracked reframing to 9:16.

API: Klap

• Endpoint: POST https://api.klap.app/v2/tasks/video-to-video
• Auth: Authorization: Bearer <KLAP_API_KEY>

Submit each clip:

import requests

headers = {
    "Authorization": f"Bearer {klap_key}",
}

# Direct file upload
with open("clip-raw.mp4", "rb") as f:
    r = requests.post(
        "https://api.klap.app/v2/tasks/video-to-video",
        headers=headers,
        files={"video": f},
        data={
            "language": "en",
            "editing_options": '{"captions":false,"reframe":true,"emojis":false,"intro_title":false}',
            "dimensions": '{"width":1080,"height":1920}'
        }
    )
task_id = r.json()["id"]
output_id = r.json().get("output_id")

Poll until ready:

# Poll every 30 seconds
r = requests.get(f"https://api.klap.app/v2/tasks/{task_id}", headers=headers)
status = r.json()["status"]  # "processing" or "ready"
output_id = r.json()["output_id"]  # project ID when ready

Export the reframed video:

# Request export
r = requests.post(
    f"https://api.klap.app/v2/projects/{output_id}/exports",
    headers=headers,
    json={}
)
export_id = r.json()["id"]

# Poll export every 15 seconds
r = requests.get(
    f"https://api.klap.app/v2/projects/{output_id}/exports/{export_id}",
    headers=headers
)
# When status != "processing", download from src_url
download_url = r.json()["src_url"]

Klap handles:

• Face detection and tracking
• Active speaker detection (for multi-person videos)
• Smooth 16:9 → 9:16 reframing
• Dynamic cropping that follows the speaker

Step 6: Add Animated Captions with Captions.ai

Upload each reframed clip to Captions.ai for professional animated captions.

API: Captions.ai (Mirage)

• Endpoint: POST https://api.mirage.app/v1/videos/captions
• Auth: x-api-key: <CAPTIONS_AI_API_KEY>

Submit each clip:

headers = {"x-api-key": captions_key}

with open("clip-reframed.mp4", "rb") as f:
    r = requests.post(
        "https://api.mirage.app/v1/videos/captions",
        headers=headers,
        files={"video": f},
        data={"caption_template_id": "ctpl_DxflLOnuKkb198FNdI9E"}
    )
video_id = r.json()["video_id"]

Poll until complete:

# Poll every 10 seconds
r = requests.get(f"https://api.mirage.app/v1/videos/{video_id}", headers=headers)
status = r.json()["status"]  # QUEUED → PROCESSING → COMPLETE or FAILED

Download the captioned video: