METHODOLOGY
How the blind listening tests work
The short version
Five voices read the same script. One is a real human. Four are AI. Real people guess blind, we publish the scores — even when the AI wins. Protocol v1.0, written 2026-09-23.
Bottom line: nobody on YouTube runs public blind A/B voice tests with the receipts attached. This page is the receipts.
Our name is a promise: claims get tested with ears, not adjectives. Every blind-test episode follows the same written protocol so you can check our work — the clips, the settings, the scores, and the dates are all on record. If we ever change the protocol, the version number changes with it, and old episodes keep their original version reference.
The format
1. Five clips, one script. Three ElevenLabs voices we generate ourselves, one official Murf sample, one real human — all reading the same ~140-word narration script.
2. Random blind labels. Clips are labeled A–E by random draw (seed logged). Nobody on camera knows which label is human until the reveal.
3. Real listeners guess. An on-camera panel and the audience both fill in score sheets: human or AI, confidence 1–5, naturalness 1–5, and what gave each clip away.
4. The reveal, then the truth. Scores are tallied and published as-is — detection rate per clip, how many were fooled, mean naturalness. If the panel spots every AI clip, the episode says so.
5. Receipts on this page. Every episode logs: model, voice, settings, generation and recording dates, the randomization seed, the panel size, and the final scores.
Clip-source labeling policy
Every clip in the series carries its true source, everywhere it appears — video, site, and score sheet:
- ElevenLabs clips: “ELEVENLABS · generated by us on [date]” — produced on our own paid Starter account (commercial license), with model, voice ID, and settings logged.
- Murf clips: “OFFICIAL MURF SAMPLE · from murf.ai — not generated by me.” We never generate Murf audio ourselves; we use samples Murf publishes on its own site, labeled verbatim.
- Human clips: “HUMAN · recorded [date] with written consent” — a real person in a real room, never a clone, never processed beyond loudness matching.
All clips are loudness-matched (−16 LUFS) before testing so volume is never a tell. No other processing is applied.
The sample texts
The script is built to include the pronunciation traps that break AI voices — $1,200, NASA, FAQ, 2026, Dr. Alvarez — inside natural narration. The reveal segment discusses each tell honestly, whichever voice it came from.
Script A — the 140-word episode script
The first time Maya checked her account, the number didn’t look real: $1,200, earned in a single weekend. She called her brother, a NASA engineer, who laughed and said the FAQ section on her website was doing the heavy lifting. By the spring of 2026, her little side project had grown into a full studio, and Dr. Alvarez — her first client — was sending referrals every single week. The secret wasn’t a bigger budget or better ads. It was her voice. Clear, calm, and consistent across every video. Viewers kept asking if she had hired a professional narrator, and she just smiled, because the truth was simpler than anyone guessed. She had found a workflow that let her sound like herself on her best day, every single day, without ever touching a microphone twice.
Script B — the 55-word Short script
Twelve hundred dollars in one weekend — that’s what Maya earned after she stopped recording herself. A NASA engineer told her the FAQ page was the real hero. By twenty twenty-six, Dr. Alvarez was her biggest fan. Five voices are about to read you this exact story. Four are AI. One is a real person. Can you hear the difference?
Scoring criteria
- Detection rate per clip — % of listeners who correctly identified each clip as human or AI.
- Fooled rate — % who guessed wrong, split into “mistook AI for human” and “mistook human for AI.”
- Mean naturalness (1–5) per clip, with the human clip as the reference point.
- Confidence-weighted accuracy — correct guesses at confidence 4–5 are “sure,” 1–3 are “unsure.”
Every result reports n (the number of listeners). A panel of 4 is a panel of 4 — never presented as a population study. On-camera panel scores and audience comment scores are tallied separately and never mixed.
Episode results
Episode 1 — “5 voices, 1 is human — can you tell?” (in production)
Status: protocol v1.0 finalized 2026-09-23. Clips not yet generated — no scores exist, and none will be published until the test is actually run. When results land, this section carries the full table: clip label → true source, detection rate, fooled rate, mean naturalness, panel size, and all logged dates.
What the protocol doesn’t prove
- A small panel can’t speak for all listeners — treat results as a documented experiment, not a census.
- The pronunciation traps in Script A test a specific skill; a voice that passes them isn’t proven on long-form, emotional, or multilingual reads.
- Official samples are a vendor’s best foot forward; our self-generated ElevenLabs clips are our own real output. The label tells you which is which so you can weight them accordingly.
Questions about the tests
Are the clips really blind?
Yes. Labels are assigned by random draw with the seed logged, the mapping stays sealed until the reveal, and the on-camera panel genuinely doesn’t know which label is human while guessing.
Do you ever re-run a bad round?
No. If a clip glitches or a round goes wrong, we say so on camera. The scores stand.
Can I take the test myself?
Yes — play along with any episode’s clips before reading the reveal, and the downloadable score sheet in each episode’s production notes works as a printout.
Why include a human at all?
Because “sounds human” is meaningless without a human on the same script. The human clip is the control every AI voice is measured against.
Generate the same script in any ElevenLabs voice and see if your friends can tell.