The Lip-Sync Checklist That Keeps AI Dialogue From Sounding Like a Bad Dub

Five things I check before a single line of dialogue leaves the script and enters a generation pipeline. The first time I put dialogue into a Lost Garden shot, I didn’t notice anything wrong for almost a full day. The line read fine. The voice was right. Then I watched the clip on my phone instead of my monitor and the mouth was landing on consonants a beat after the sound hit. Not much. Maybe eighty milliseconds. Enough that a friend who watched it cold asked if the video was buffering. That gap is the whole problem with AI dialogue, and it’s smaller than people think. Human perception of lip-sync error isn’t measured in seconds, it’s measured in tens of milliseconds. The ITU’s own broadcast standard (BT.1359) puts the detectable threshold at audio leading video by more than 45ms, or lagging it by more than 125ms. Film post houses hold themselves to an even tighter bar, often citing 22ms in either direction as the ceiling for “acceptable.” For comparison, a single frame at 24fps is about 42ms. You can blow the tolerance with less than two frames of drift. Nobody tells you this when they hand you a voice model and a lip-sync tool and say “go make a talking character.” So here’s the checklist I actually run now, after that phone-screen moment made me stop trusting the monitor. 1. Write dialogue for a mouth that has no idea what a scene is A script written for a human actor assumes breath, hesitation, and the physical reality of a body delivering a line while thinking about the next one. A text-to-speech model has none of that context. It reads what’s on the page, and only what’s on the page. This means: Short lines survive better than long ones. Most voice models start losing natural rhythm past two sentences without explicit pacing marks. Punctuation is direction, not decoration. A comma is a breath. An ellipsis is a hesitation. If you don’t mark where the character pauses, the model picks a rhythm for you, and it’s usually flatter than what you wanted. Say the line out loud before you type it as a prompt. If it doesn’t sound like something a person would actually say, no amount of lip-sync polish fixes that afterward. 2. Lock the voice before you lock the shot I made the mistake early on of generating a face, generating a voice, and only then checking whether they matched. They didn’t. The voice I’d picked sounded like it belonged to someone ten years older than the face on screen. Treat voice like you’d treat a character’s visual reference: decide it once, early, and reuse it deliberately. ElevenLabs’ voice cloning splits into two tiers worth knowing about: Instant Voice Cloning, which works from under a minute of sample audio, and Professional Voice Cloning, which needs several hours of clean source material but holds up far better across long projects. If a character is going to speak more than a handful of lines across your whole production, the professional clone is worth the extra setup time. It’s the difference between a voice that’s consistent and one that’s merely similar . The voice is not a detail you fix in the mix. It’s a casting decision, and it happens before the camera does anything. 3. Separate “does the voice sound real” from “does the mouth match the voice” These are two different engineering problems, and most tools are only good at one of them. It took me longer than it should have to stop expecting a single tool to nail both. Voice realism (breath, tone, emotional inflection, accent) is where models like ElevenLabs lead. Its Eleven v3 model added multi-speaker dialogue and audio tags for expressive delivery, which matters if a scene has more than one character talking. Sync accuracy (does the mouth shape match the phoneme at the right millisecond) is a narrower, harder problem that dedicated lip-sync tools chase specifically. Sync Labs, for instance, is built around a developer-facing API whose whole pitch is tighter mouth-to-audio alignment than general-purpose video tools manage. HeyGen, aimed more at presenter-style video, advertises sync accuracy in the range of hundredths of a second on its avatar pipeline. Pick tools by which problem you’re solving this shot , not by brand loyalty. A gorgeous voice performance riding on sloppy sync reads worse than a flatter voice that’s locked to the frame. 4. Feed matched duration, not approximate duration This is the one nobody warns you about until it bites you. Lip-sync tools align mouth movement to an audio track’s actual waveform, not to your intended runtime. If your voice line runs 3.4 seconds and your video clip runs 3.0, something has to give: either the video gets stretched (and everything in the shot subtly slows down, not just the mouth), or the audio gets clipped (and you lose the end of a word). Generate the voice line first, measure its exact duration, then generate or trim the video clip to that number. Not the other way around. I learned this after a Lost Garden line got cut off mid-word because I’d built the shot to a “close enough” length and let the lip-sync tool force the fit. 5. Watch it once with no sound, then once at full volume on a phone Two separate checks, because they catch two separate failures: Mute it and watch the performance. Does the face read as a person thinking and reacting, independent of what’s coming out of the speakers? If the acting doesn’t hold up silent, no amount of sync accuracy saves it. Play it with sound on a phone speaker, not your studio monitor. Compressed audio and small speakers expose timing drift that a good pair of headphones smooths over. That eighty-millisecond gap I mentioned at the start was invisible on my desktop setup and obvious on a five-inch screen. 6. Keep a dialogue log, the same way you’d log a seed If you’ve read anything else I’ve written about AI filmmaking, you already know I’m going to say this: write down what worked. For every line of dialogue that ships, I keep the voice model and version, the exact audio file, the lip-sync tool and its settings, and which take got approved. Voice models update. Lip-sync tools change their defaults. Without a log, a line that synced perfectly last month can quietly stop matching after a silent model update, and you won’t know why until a viewer tells you the mouth looks off again. That log lives next to the shot plan for Lost Garden, the same place the character bible and camera notes sit. Keeping dialogue continuity next to visual continuity turned out to matter more than I expected. A scene isn’t just a face that stays the same. It’s a voice that stays the same too. None of this is complicated once you’ve been burned by it once. The tolerance window is small, the tools are genuinely good at narrow slices of the problem, and the fix for almost every failure mode above is sequencing: write for the ear, lock the voice, separate the two engineering problems, match durations exactly, check twice on the right hardware, and log what you did. Most of this is quality control, not craft. But a bad half-second of dialogue is the fastest way to remind a viewer they’re watching something synthetic, so I’d rather catch it on my own phone screen than have someone else point it out first.