Why do all-in-one transcript tools break at the caption layer

They break there because the caption layer is not a copy of the speech. Across five public Reels, 169 words were spoken in the first ten seconds and only 29 of them ever appeared on screen: 17.2% coverage. A tool that reads the caption layer returns a headline. A tool that reads the audio returns the words but drops the names that were only ever printed.

“Caption layer” here means the text burned into the picture, the part a screenshot would capture. It is not Instagram’s in-app caption setting, and it is not the post description. Those are separate layers and this page does not test them.

What was counted

Five public Reels, first ten seconds each, one frame per second, fifty frames. The burned-in text was read off those frames directly. The same ten seconds of audio was transcribed locally with faster-whisper (tiny, revision d90ca5fe, CPU int8, one thread, English), and every word carried a start time, so a word counted only if it began before the ten-second mark. Words were compared after lowercasing and stripping punctuation. “Spoken words” below means distinct words, so a repeat counts once: the stricter reading.

Read the last column as one question: if a tool copied the screen perfectly, how much of the speech would it recover? Split the sample first. Two Reels carried captions that followed the voice — a summary card in Ddux1OTiIn7, word by word in Ddv4nWFjTiB — and on those 60 spoken words the screen carried 27, or 45.0%. Three Reels carried readable text that was never spoken: a title card and a promo line, a lower third, one headline. They are not unreadable, but nothing on screen was said aloud, so they sit outside the coverage denominator and are marked no-speech-caption. Averaging all five gives 17.2%, which describes a mixed sample rather than a caption layer. And this is one recogniser on one ten-second window per Reel: it measures what the screen held, not what every all-in-one tool does.

The other direction is worse

The screen also carried 35 distinct words never spoken in that window: 54.7% of everything printed — NORTHEAST CAUSES TRAVEL DISRUPTIONS, ANDREW DYMBURT, TENNIS STAR TAYLOR TOWNSEND ON EMERGENCY APPENDECTOMY. A caption-reading tool does not return a shortened transcript here; it returns a title.

The one Reel that did it properly

Only Ddv4nWFjTiB used word-by-word captions that followed the voice, and even it stopped at 53.1%. It kept I think a murder, on this podcast, a terribly nice man, nobody would, sitting next to you, and dropped ten spoken words — one of them a person’s name, Ed Davie, which no frame showed. Even the best caption layer here is a paraphrase with the proper nouns missing, the exact category a recogniser is most likely to spell wrong alone.

Run the same check on your own Reel

Five steps. One, take the first ten seconds, where the hook card sits. Two, pull one frame per second and crop the bottom half. Three, read the text off each frame as plain words. Four, transcribe the same ten seconds with any recogniser that returns word times, keeping words that start before ten seconds. Five, lower-case both lists, drop duplicates, and divide the overlap by the spoken count.

Fix the pass mark before you look: 90% or better can stand in for a transcript, under half cannot. On that scale the caption-bearing Reels at 45.0% still fail.

What to do with that

Use the audio for the words and the screen for the names. Where they disagree, the screen is usually right about spelling, because a person typed it, and the audio is right about everything else. If a tool offers one layer and calls it a transcript, ask which layer it read.

Scope and limits

Five Reels, first ten seconds only. One frame per second can miss a caption shown for less than a second, which pushes coverage down, not up. Matching ignores case and punctuation, so Its still counts for It’s. A small recogniser on a laptop CPU can mishear, but that does not explain two Reels scoring zero, where nothing printed matched anything spoken. Nothing here tests the in-app caption feature, private accounts, or marketing claims.

Questions people actually ask

Why does my all-in-one tool return a headline instead of a transcript?

Because it read the caption layer, and here the caption layer held a headline. It never looked at the audio. Ask for the audio route and the same link usually returns words.

Does Instagram’s own auto-caption switch change this measurement?

No. This test read text burned into the picture, which is present whether that feature is on or off. The in-app layer is a different thing to measure.

Why is coverage 17.2% rather than zero?

One Reel used word-by-word captions and contributed 17 of the 29 matched words. Without it, the other four score 12 of 137 spoken words: 8.8%.

Is the caption layer useless then?

Not useless, just not a transcript. It is the layer that spells names correctly. Check your recogniser’s output against the on-screen name before quoting.

How long does the check take on one Reel?

About ten minutes: one to pull ten frames, five to read them, the rest to transcribe ten seconds and compare two short lists.

Does this mean audio transcription is always the better route?

For spoken words, yes. For names, numbers and titles it is weaker, and the caption layer is the cross-check. The two fail in opposite directions.