Back to getSRT
getSRT Blog/ A Practical Video-to-Captions Workflow with whisper.cpp

A Practical Video-to-Captions Workflow with whisper.cpp

Extract audio from video, generate SRT/VTT captions, and reformat them to meet accessibility line-length and timing guidelines.

2 min read
  • accessibility
  • subtitles
  • guide

The workflow

Step 1 — Extract audio from the video

ffmpeg -i video.mp4 -ar 16000 -ac 1 -c:a pcm_s16le audio.wav

Step 2 — Generate both SRT and WebVTT

./whisper-cli -m models/ggml-small.en.bin -f audio.wav -osrt -ovtt --output-file captions
  • .srt is the most widely-supported format (YouTube, Vimeo, most video editors).
  • .vtt is what HTML5 <track> elements expect natively in the browser.

Step 3 — Enforce accessibility line-length guidance

Raw whisper.cpp segments don't respect caption readability guidelines out of the box. A reformatting pass wraps text to a max line length and checks reading speed:

import re
 
MAX_CHARS_PER_LINE = 37
MAX_READING_CHARS_PER_SEC = 20  # rough WCAG-friendly reading speed guideline
 
def wrap_caption(text: str) -> str:
    words = text.split()
    lines, current = [], ""
    for word in words:
        if len(current) + len(word) + 1 > MAX_CHARS_PER_LINE:
            lines.append(current)
            current = word
        else:
            current = f"{current} {word}".strip()
    if current:
        lines.append(current)
    return "\n".join(lines[:2])  # cap at 2 lines per caption, standard convention
 
def check_reading_speed(text: str, duration_sec: float) -> bool:
    if duration_sec <= 0:
        return False
    return (len(text) / duration_sec) <= MAX_READING_CHARS_PER_SEC

Step 4 — Flag captions that are shown too briefly

def validate_captions(captions):
    warnings = []
    for cap in captions:
        duration = cap["end"] - cap["start"]
        if duration <= 0:
            warnings.append(f"Error: caption at {cap['start']}s has zero/negative duration")
            continue
        if not check_reading_speed(cap["text"], duration):
            warnings.append(f"Warning: caption at {cap['start']}s may be too fast to read comfortably")
    return warnings

Step 5 — Attach to an HTML5 video

<video controls>
  <source src="video.mp4" type="video/mp4">
  <track kind="captions" src="captions.vtt" srclang="en" label="English" default>
</video>

Error handling for the pipeline script

set -euo pipefail
 
if [ ! -f video.mp4 ]; then
  echo "Error: video.mp4 not found" >&2
  exit 1
fi
 
ffmpeg -y -loglevel error -i video.mp4 -ar 16000 -ac 1 -c:a pcm_s16le audio.wav \
  || { echo "Error: audio extraction failed" >&2; exit 1; }
 
./whisper-cli -m models/ggml-small.en.bin -f audio.wav -osrt -ovtt --output-file captions \
  || { echo "Error: caption generation failed" >&2; exit 1; }

Up next

The final article covers advanced accessibility topics: multi-language subtitle tracks, audio descriptions for visually-impaired viewers, and automated QA for caption-video sync drift.