Whisper Tiny vs Turbo for AI Voiceover Captions: Tiny Wrote "Paths" for "Pass"

Disclosure: this post has no affiliate links. The shop page linked at the end is our own product list and may contain affiliate links; we may earn a commission if you buy through those. This is our own test log from October 28, 2026 (KST). No tool vendor provided or reviewed it.

Short answer from one small test: for English captions of a clean AI voiceover, the smallest Whisper model was not enough, but the second smallest nearly was. We ran four local Whisper sizes (tiny, base, small, large-v3-turbo) three times each on two 21–22-second takes of the same 73-word narration. Small and large-v3-turbo matched the script word for word on both takes. Base matched one take and split "voiceover" into "voice over" on the other. Tiny made the same 3 word errors on both takes, and one of them changed the meaning: "If all three pass" became "If all three paths". Speed did not decide anything here: every model finished each clip in under 6 seconds on our laptop.

The question

You make short videos with an AI voiceover and generate captions locally with Whisper. Bigger models are slower to download and load, so it is tempting to use the smallest one. Which size is enough for a clean English voiceover, and what kind of mistakes does a too-small model make?

What we ran

  • Audio: the two takes from our October 27 local voice test (22.08 s and 21.44 s, VoxCPM2, a model-generated voice, not a real person). No new audio was made. The script starts "Before you export a tutorial, check three things." and has 73 words.
  • Models: MLX conversions of Whisper tiny, base and small (the multilingual versions, not the English-only .en ones) and large-v3-turbo, through mlx-whisper 0.4.3.
  • Same settings for all: language fixed to English, no prompt or word list, no editing of the output.
  • Runs: per model and take, one warm-up run that we did not time, then 3 timed runs. All 3 runs gave identical text every time, so the error counts below are the same across runs.
  • Counting: we lowercased both texts, removed punctuation, and counted word edits (wrong word, extra word, missing word) against the 73-word script.
  • Machine: Apple M5 Pro laptop, 24 GB memory; MLX ran on the Apple GPU.

Results

ModelWord errors, take 1Word errors, take 2What went wrongTimed runs per clip (s)
tiny33"voiceover" → "voice over" (2 edits); "pass" → "paths"1.2–1.5
base02take 2 only: "voiceover" → "voice over"1.0–1.9
small00matched the script0.7–5.4
large-v3-turbo00matched the script0.6–0.9

The "voice over" split counts as 2 edits under our rule (one changed word plus one extra word), but a viewer would barely notice it. The "paths" error is different. It is a real word, it reads naturally, and it changes the instruction. That is the kind of caption mistake a quick skim misses.

The timing column is a range over 3 runs, and it was not consistent. Small took 5.0–5.4 s on take 1 but 0.7–0.9 s on take 2, and large-v3-turbo was faster than tiny in this run. We did not find out why, so do not read the timing as a speed ranking. What it does show is that on this laptop, every one of the four sizes finished a 22-second clip quickly.

What we decided

For our English voiceover captions we keep large-v3-turbo. It got both takes right, and on this machine it was not slower once loaded. If you must use a smaller model, base got one take fully right, but tiny made a meaning-changing error on both takes, so we would not publish tiny captions without reading every line against the script.

A clean script is also not a guarantee of clean captions. In our October 22 caption check on a Korean narration, the large-v3-turbo caption came back with a noun and its particle merged into a wrong word; we could not tell whether the voice or the model caused it. English and Korean behave differently; check your own language.

What this does not show

  • One 73-word script, two takes of one model-generated voice, clean audio with no music or noise. Real recordings, accents and background sound were not tested.
  • We did not test the English-only tiny.en and base.en models. OpenAI's README says those "tend to perform better" for English, so they may close part of the gap.
  • Medium and the full large-v3 were not run. Timing was measured on one laptop with MLX and was inconsistent between runs; no CPU-only or other-GPU timing. OpenAI's README lists tiny as faster than turbo (~10x vs ~8x relative speed), which is another reason not to treat our timing as a ranking.
  • Word errors only. We did not check caption timing (where each line starts and ends).

Repeat it on your own voiceover

  1. Keep the exact script you recorded or generated as a plain text file.
  2. Transcribe the same audio with two sizes, for example mlx_whisper take.wav --model mlx-community/whisper-tiny --language en and the same with mlx-community/whisper-large-v3-turbo (or the openai-whisper CLI with --model tiny and --model turbo).
  3. Compare each transcript with your script word by word. Look hardest at short words that are still real words after a mistake, like pass/paths.
  4. Pick the smallest model that matches your script on two or more takes, then still read the final captions once in the video.

Sources checked

  • openai/whisper README on GitHub (model sizes: tiny 39M, base 74M, small 244M, turbo 809M parameters; note that .en models tend to perform better for English; performance varies by language), read October 28, 2026
  • Our raw measurements: model snapshots, input file hashes, every transcript and timing, kept in our production log.

Other things we link to, with buying checklists, are on our Label Picks shop page.

Comments

Post a Comment

Popular posts from this blog

FFmpeg Shows −1.5 dBTP. Did You Actually Fix the Audio?

An output_tp of −1.50 in an FFmpeg log does not mean your original audio has been fixed. A command can show that number without saving any corrected audio. Before uploading, measure the file you actually exported. This follow-up to our AI song quality checklist tackles one practical mistake: confusing filter output statistics with a saved deliverable. You need FFmpeg installed and a local audio file. These commands process files on your computer; they do not upload your recording. One recording, two different stages On September 18, 2026, we measured one newly generated, roughly 2-minute-20-second MP3. Its integrated loudness was −11.41 LUFS and its true peak was +0.84 dBTP . The same analysis log showed output_tp: -1.50 . input_i and input_tp describe the file supplied to that command. output_i and output_tp describe the signal after the filter in that execution. They do not establish the measured properties of a separately encoded final file. We then reduced gain by...
Read more

VoxCPM2 vs ElevenLabs for a 30-Second Science Short: We Timed Both on the Same Script

Image
Disclosure: DigitalHarvest (NSST) is an independent affiliate of ElevenLabs. We may earn a commission if you subscribe through the referral link in this article. ElevenLabs has not sponsored, reviewed, or approved this article. This relationship does not change the numbers below, and the free local option wins on cost in our own test. We make 30-second science Shorts from public-domain NOAA ocean footage, and every one needs about six lines of narration. On October 4, 2026 we took one finished script and voiced the same six lines twice: once with VoxCPM2, an open-source model running on our own Apple Silicon Mac, and once with ElevenLabs on the paid Creator plan we already use. Then we ran both outputs through the same Whisper check our pipeline uses before anything is published. Here is what came out. The script we used The episode is "Why doesn't hot water on the seafloor boil?" (a small black smoker vent filmed at 2,947 m by NOAA Ocean Exploration). Six lines, 428...
Read more

Before You Upload an AI Song or Video: The Checks We Run (Free Commands)

The FFmpeg and file-hash commands below run locally. No purchase is needed to follow them. In September 2026 we published a 140-second "music video" for one of our AI songs. It was one still picture with a small moving waveform along the bottom. Our pre-upload checks had measured the length, the resolution and the loudness. Nobody had checked whether the picture actually moved. We deleted it from our channel the next morning. This post shows the checks we run now. Every command below runs on your own computer with ffmpeg , and every number comes from files we measured ourselves. 1. Loudness and true peak (free, one command) ffmpeg -hide_banner -nostats -i song.mp4 -af ebur128=peak=true -f null - Read the Summary block at the end: I: is the integrated loudness in LUFS. Peak: under "True peak" is the true peak. ffmpeg prints the unit as dBFS. A test file we deliberately pushed too loud read I: -8.4 LUFS and Peak: 3.5 . Our own working baseline is ...
Read more