fqmpeg Subtitles & Captions: Burn, Extract, Embed

fqmpeg's C3 cluster is four verbs that cover every "deal with subtitles" task: subtitle and subtitle-burn hardcode captions into the video pixels, extract-subtitle pulls a subtitle track out of a container as .srt, and sidecar embeds an external .srt/.ass/.vtt as a selectable track inside an MKV. Together they handle the round-trip — read what's already in the file, write something new in or beside it.

This guide walks through each command — the FFmpeg flags it generates, defaults, output naming — then chains them into real workflows including the Whisper → fqmpeg pipeline. Everything below is verified against the source in src/commands/ of fqmpeg 3.0.1.

What you'll get out of this guide

• Which of the 4 verbs to pick (decision matrix: hard vs. soft subs, read vs. write)
• Exact FFmpeg invocation each verb generates (verified --dry-run output)
• Defaults, allowed values, and output filenames for every command
• Three end-to-end recipes including the Whisper auto-subtitle pipeline

Subtitles in fqmpeg: Hard vs. Soft, Read vs. Write

Before the verb-by-verb walkthrough, internalize one mental model: every subtitle decision splits along two axes.

Axis 1 — hard vs. soft subs. Hard subs (also called "burned in" or "open captions") are baked into the video pixels. They cannot be turned off, restyled, or re-translated; they ride with the picture forever. Soft subs are stored as a separate track in the container; players can toggle them on and off, and YouTube/Netflix-style multi-language menus rely on them.

Axis 2 — read vs. write. Some verbs add subtitles to a file (subtitle, subtitle-burn, sidecar); one extracts an existing track out (extract-subtitle).

Three things to know before reading on:

1. Both subtitle and subtitle-burn are hard subs. The only difference is style controls. Despite the name, subtitle does not produce a soft track — for that, use sidecar.
2. Soft subs need MKV. MP4 technically supports mov_text subtitles, but compatibility across players is patchy. fqmpeg's sidecar defaults to .mkv for the same reason most archival pipelines do — Matroska is the most permissive container for multi-track subs.
3. extract-subtitle is text-only by default. It outputs .srt, which means image-based subs (PGS / DVDSub / VobSub from Blu-ray and DVD rips) can't be extracted directly — they're a different beast that requires OCR. For text tracks (most modern files), it's a one-liner.

If you don't have a subtitle file to work with yet, the FFmpeg + Whisper auto-subtitles guide shows how to generate one from the audio of any video. Recipe 3 below chains that pipeline directly into subtitle.

Hard-Subbing: subtitle and subtitle-burn

Both verbs use FFmpeg's subtitles filter, which decodes the subtitle file with libass and renders it onto the video before encoding. The output is a normal video file with the captions painted permanently into the pixels — no separate track, no toggle.

subtitle — Hardcode with style controls

The full-featured burn-in. Choose font size, color (5 presets), and on-screen position (top / center / bottom). Audio is stream-copied so only the video re-encodes.

• Source: src/commands/subtitle.js
• Filter: subtitles=<path>:force_style='FontSize=N,PrimaryColour=…,Alignment=…'
• Audio: -c:a copy (no re-encoding)
• Path escaping: the subtitle path is auto-resolved to absolute and escaped (: becomes \:, backslashes flipped to forward slashes) — required by libavfilter on Windows and inside complex filter graphs

$ npx fqmpeg subtitle input.mp4 captions.srt --dry-run

  ffmpeg -i input.mp4 -vf subtitles=/abs/path/to/captions.srt:force_style='FontSize=24,PrimaryColour=&H00FFFFFF,Alignment=2' -c:a copy input-subtitled.mp4

$ npx fqmpeg subtitle input.mp4 captions.srt --font-size 32 --color yellow --pos top --dry-run

  ffmpeg -i input.mp4 -vf subtitles=/abs/path/to/captions.srt:force_style='FontSize=32,PrimaryColour=&H0000FFFF,Alignment=8' -c:a copy input-subtitled.mp4

The color names map to ASS/SSA's BGR hex format internally — white = &H00FFFFFF, yellow = &H0000FFFF, red = &H000000FF, etc. (Note the channel order is BGR, not RGB; that's why "red" is 0000FF and "cyan" is FFFF00.) If you need a custom color, take the --dry-run output, replace the hex, and run the FFmpeg directly.

For position, the numbers are libass alignment values: 1–3 are bottom-left/center/right, 4–6 center vertically, 7–9 are top. fqmpeg exposes the three most common (bottom = 2, top = 8, center = 5); for left/right alignment, drop down to FFmpeg.

subtitle-burn — Same burn-in, simpler API

Identical effect to subtitle but with no styling beyond font size. Useful when you trust the subtitle file's own styling (typical for .ass files that ship their own colors and positions) or just want to bake captions in without thinking about it.

• Source: src/commands/subtitle-burn.js
• Filter: subtitles=<path>:force_style='FontSize=N'
• Path escaping: none — the path is passed verbatim. Use simple relative paths (captions.srt) and run from the same directory, or pass an .ass file with its own styles

$ npx fqmpeg subtitle-burn input.mp4 captions.srt --dry-run

  ffmpeg -i input.mp4 -vf subtitles=captions.srt:force_style='FontSize=24' -c:a copy input-burned.mp4

$ npx fqmpeg subtitle-burn input.mp4 captions.srt --font-size 28 --dry-run

  ffmpeg -i input.mp4 -vf subtitles=captions.srt:force_style='FontSize=28' -c:a copy input-burned.mp4

Which to pick: subtitle is the safer default (path escaping + style controls, defaults are sensible). Reach for subtitle-burn when you want a one-token name and trust the subtitle file's own styling — typical for fan-translated .ass files where re-styling would override the translator's intent.

Reading Existing Tracks: extract-subtitle

extract-subtitle — Pull a track out as .srt

Selects a single subtitle stream from a multi-track file (typically MKV from a Blu-ray rip or a multi-language YouTube download) and writes it to a standalone .srt. Stream-copy mode means it's near-instantaneous regardless of input size.

• Source: src/commands/extract-subtitle.js
• Mapping: -map 0:s:<n> -c:s srt
• Output: SubRip text format (.srt). Image-based subs (PGS, DVDSub) won't transcode to SRT — that's an OCR problem, not a remux problem

$ npx fqmpeg extract-subtitle movie.mkv --dry-run

  ffmpeg -i movie.mkv -map 0:s:0 -c:s srt movie-sub.srt

$ npx fqmpeg extract-subtitle movie.mkv --stream 1 -o english.srt --dry-run

  ffmpeg -i movie.mkv -map 0:s:1 -c:s srt english.srt

To list which subtitle streams a file contains, run ffprobe -v error -show_streams input.mkv | grep -E "index=|codec_type=subtitle|TAG:language" or use fqmpeg's info verb (covered in the hub guide). The output gives you the mapping: s:0 = jpn, s:1 = eng, etc., so you know which --stream <n> to pass.

If extract-subtitle errors with Subtitle encoding currently only possible from text to text or bitmap to bitmap, the source is a bitmap subtitle (PGS from Blu-ray, VobSub from DVD). You can either copy it bit-for-bit (drop -c:s srt, use -c:s copy, output to .sup or .idx/.sub) or run it through a separate OCR tool like pgs2srt or Subtitle Edit.

Soft-Embedding: sidecar

sidecar — Add an external .srt as a selectable track

Takes a video and a subtitle file, copies both into a single MKV with the subtitle as a toggleable track (default off in most players, but selectable from the player's menu). No re-encoding — video and audio stream-copy, subtitles get muxed in.

• Source: src/commands/sidecar.js
• Codec: -c copy -c:s srt (video/audio passthrough, subtitles stored as SRT)
• Maps: all video streams from input 0, all audio from input 0, subtitle from input 1
• Output container: MKV by default. MP4 can technically hold one mov_text track, but adding multiple soft subs or non-Latin scripts is unreliable across players

$ npx fqmpeg sidecar input.mp4 captions.srt --dry-run

  ffmpeg -i input.mp4 -i captions.srt -c copy -c:s srt -map 0:v -map 0:a -map 1:0 -metadata:s:s:0 language=und input-subtitled.mkv

$ npx fqmpeg sidecar input.mp4 captions.srt --language en --title "English" --dry-run

  ffmpeg -i input.mp4 -i captions.srt -c copy -c:s srt -map 0:v -map 0:a -map 1:0 -metadata:s:s:0 language=en -metadata:s:s:0 title=English input-subtitled.mkv

The --language value sets the track's metadata so VLC, MPV, mpv-android, Plex, Jellyfin, etc. show the right name in the audio/sub menu. Most players use ISO 639-2 three-letter codes (eng, jpn, spa); two-letter ISO 639-1 (en, ja, es) often works too because libavformat normalizes them, but stick to the three-letter form for maximum compatibility. Use und (the default) when the language genuinely isn't known.

To embed multiple soft subs (e.g., English + Japanese in the same MKV), run sidecar once and then re-run on the output with the second .srt:

npx fqmpeg sidecar input.mp4 en.srt --language eng --title "English" -o step1.mkv
npx fqmpeg sidecar step1.mkv jp.srt --language jpn --title "日本語" -o final.mkv

The MKV produced by step 1 is a valid input for step 2; the second pass preserves the existing English track and adds the Japanese one.

Real-World Recipes

Each recipe chains multiple verbs into a real workflow.

Recipe 1: Whisper → fqmpeg auto-burn pipeline

You have a tutorial screencast and want hardcoded English captions without typing a word of subtitle yourself. Whisper transcribes the audio; subtitle burns the result.

# Step 1: extract audio for Whisper (16 kHz mono is enough)
ffmpeg -i screencast.mp4 -ac 1 -ar 16000 audio.wav

# Step 2: run Whisper to produce SRT
whisper audio.wav --model small --output_format srt
# → audio.srt

# Step 3: burn into the original video with readable styling
npx fqmpeg subtitle screencast.mp4 audio.srt --font-size 28 --color white --pos bottom
# → screencast-subtitled.mp4

The full pipeline takes 2–3× the video duration on a CPU-only laptop with --model small (faster on a GPU, slower on --model large). For batch processing or production accuracy, the FFmpeg + Whisper auto-subtitles guide wraps the three steps in a Python script and covers model choice, language detection, and quality tuning.

Recipe 2: Multilingual MKV from a YouTube download

You used yt-dlp to grab a video that has English audio plus auto-generated .vtt captions in English, Japanese, and Spanish (yt-dlp --write-subs --sub-langs "en,ja,es" --convert-subs srt URL). You want a single MKV that carries the original video plus all three languages as soft subs.

# Step 1: embed English first
npx fqmpeg sidecar video.mp4 video.en.srt --language eng --title "English" -o step1.mkv

# Step 2: add Japanese
npx fqmpeg sidecar step1.mkv video.ja.srt --language jpn --title "日本語" -o step2.mkv

# Step 3: add Spanish
npx fqmpeg sidecar step2.mkv video.es.srt --language spa --title "Español" -o final.mkv

Open final.mkv in VLC or MPV and the subtitles menu lists all three languages. Total runtime is seconds because every step is stream-copy. For Plex / Jellyfin libraries, the language tags also drive auto-subtitle selection — set your default audio/sub language in the server, and the right track lights up on each device.

Recipe 3: Re-style burned subs from a Blu-ray rip

You have a .mkv rip with text-based English subtitles you want to bake into the video, but at a larger size and yellow color (the rip's default styling is too small on a TV across the room).

# Step 1: pull the English subtitle out as SRT
npx fqmpeg extract-subtitle source.mkv --stream 0 -o english.srt

# Step 2: burn it back in with bigger, yellow text
npx fqmpeg subtitle source.mkv english.srt --font-size 32 --color yellow --pos bottom
# → source-subtitled.mkv

Step 1 is instant (stream-copy), step 2 re-encodes the video. If your TV is 4K and the source is 1080p, this is also a good moment to chain in compress at a higher bitrate, since you're already re-encoding. If the rip's subtitles are bitmap (PGS), step 1 fails — fall back to OCR with pgs2srt first, then continue from step 2.

Frequently Asked Questions

Why doesn't subtitle produce a soft (toggleable) track despite its name?

Because under the hood it uses FFmpeg's subtitles filter, which renders captions onto the video as part of the encode. The verb name is the user-facing intent ("subtitle this video") rather than the technical mechanism. For a soft, toggleable track use sidecar instead — that's the one that muxes the subtitle file as a stream rather than burning it.

What's the actual difference between subtitle and subtitle-burn?

Both produce hardcoded captions using the same FFmpeg filter. subtitle adds three style controls (--font-size, --color, --pos) and auto-escapes the subtitle file path so it works with absolute paths, paths with spaces, or Windows drive letters. subtitle-burn only exposes --font-size and passes the path verbatim. Use subtitle unless you specifically want the simpler form to preserve .ass self-styling.

Why does the path escaping matter?

The libavfilter parser interprets : as an option separator inside filter graphs. So a path like C:\Users\me\sub.srt would be parsed as filter "C" with option "Users", which fails immediately. subtitle resolves the path to absolute, replaces backslashes with forward slashes, and escapes colons as \: — all in one step. If you've ever pasted an FFmpeg subtitles= command from Stack Overflow and gotten Unable to open errors on Windows, this is why.

Can I burn .ass subtitles with their own styling?

Yes — .ass (Advanced SubStation Alpha) files carry their own font, color, and positioning data, and libass renders them faithfully. subtitle-burn is the cleaner choice here because passing --color to subtitle would override the file's per-line styling. Just make sure the fonts referenced in the .ass are installed on the system running FFmpeg, or libass falls back to a default.

My extracted SRT only has timecodes, no text. What happened?

Almost always one of two things. (1) The source subtitle was bitmap (PGS / VobSub) and FFmpeg fell back to writing empty SRT entries — check the input format with ffprobe -show_streams and look for codec_name=hdmv_pgs_subtitle or dvd_subtitle. (2) The source SRT had encoding issues; try forcing UTF-8 with iconv -f WINDOWS-1252 -t UTF-8 input.srt > output.srt before extracting. SRT itself is plain text, so once you have a clean text track in, the round-trip is lossless.

Does sidecar work with MP4 instead of MKV?

In theory yes — MP4 supports mov_text for one subtitle track. In practice, fqmpeg defaults to MKV because mov_text drops styling, struggles with non-Latin scripts, and isn't recognized by every player (mobile Safari is particularly inconsistent). If you must ship MP4, pass -o output.mp4 and accept the limitations. For everything else, MKV plays in VLC, MPV, Plex, Jellyfin, modern smart TVs, and most browser-based players via hls.js fragmentation.

Why is --language jpn a three-letter code, not ja?

The Matroska spec recommends ISO 639-2 (three-letter) codes for the Language element, and most player implementations expect them. fqmpeg's default und (undefined) is the official ISO 639-2 placeholder for "language not declared." libavformat is lenient and usually accepts ISO 639-1 (ja, en, es) too, but the long form is what shows up in tools like mediainfo, MakeMKV, and Plex's auto-selection logic, so stick to it.

How do I batch-burn captions across a folder of videos?

Standard shell loop. Assuming each video has a matching .srt (e.g., lesson-01.mp4 + lesson-01.srt):

for v in lessons/*.mp4; do
  srt="${v%.mp4}.srt"
  [ -f "$srt" ] || { echo "skip: $v (no srt)"; continue; }
  npx fqmpeg subtitle "$v" "$srt" --font-size 28
done

Each output lands next to its input as lesson-01-subtitled.mp4. The skip line keeps the loop from failing if a video is missing its SRT.

Wrapping Up

The four C3 verbs cover the full subtitle round-trip:

• subtitle for hardcoded captions with style controls (the default choice)
• subtitle-burn for hardcode-only when the file's own styling should win
• extract-subtitle for pulling existing tracks out of MKV/MP4 as SRT
• sidecar for soft-embedding an external SRT into MKV as a selectable track

Every verb prints its underlying FFmpeg invocation under --dry-run, so you can copy it, adapt it (custom hex color, ISO 639-3 language code, MP4 output), or learn the syntax behind the verb. For automated subtitle generation from audio, the Whisper auto-subtitles guide covers the upstream pipeline. For the broader fqmpeg map, return to the fqmpeg complete guide.