AI voice & lipsync

The voice pipeline produces character speech audio plus the matching facial / blendshape animation in one orchestrated pass. Three components ship in 1.0:

• ElevenLabs — leading TTS provider (BYOK).
• NVIDIA Audio2Face — high-quality blendshape-driving from audio (local SDK or cloud).
• uLipSync — fast viseme-based lipsync, fully local, OSS (MIT, absorbed).

Tool family: voice_* — 40 tools.

When to use which

Need	Use
Hero dialogue, photoreal facial	Audio2Face
NPC chatter, mobile target	uLipSync
One voice, many lines	ElevenLabs + Audio2Face
Offline, no API budget	Sentis-local TTS + uLipSync

You can mix — most projects use ElevenLabs for the voice and uLipSync at runtime for cheap viseme matching, with Audio2Face baked offline for cinematics.

A round-trip in chat

Generate dialogue for the Wizard NPC saying "I have foreseen your arrival, traveler".
Voice should be old, gravelly. Drive the Wizard's blendshapes with the audio.
Save as Assets/Audio/Wizard_Line_001.wav and Assets/Animations/Wizard_Line_001.anim.

The voice-author specialist:

1. voice_elevenlabs_generate --voice old_gravelly_male --text "..." → produces Wizard_Line_001.wav
2. voice_audio2face_drive --audio Wizard_Line_001.wav --target Wizard.HeadRig → produces blendshape animation curves
3. animation_clip_save --path Assets/Animations/Wizard_Line_001.anim

Total time: 15-60 seconds depending on dialogue length.

ElevenLabs

Wire your ElevenLabs API key under Settings → Providers → ElevenLabs. Buril supports:

• All v1/v2 voices (preset + cloned).
• SSML mark-up for emphasis, pauses, prosody.
• Streaming generation (for long dialogue, audio plays as it's generated).

Tools:

Tool	Purpose
voice_elevenlabs_generate	Text → audio
voice_elevenlabs_list_voices	Enumerate available voices
voice_elevenlabs_clone_voice	Clone a new voice from sample audio (requires Creator tier or higher)
voice_elevenlabs_dub	Translate + voice-match an existing audio clip

Costs are pay-per-character. The bridge does not throttle — you set quota limits on the ElevenLabs side.

NVIDIA Audio2Face

Audio2Face ships in two forms:

• Local SDK — requires an Omniverse install and an RTX GPU. Best quality, no per-call cost after install.
• Cloud (Audio2Face-3D) — NVIDIA's hosted API, BYOK, pay-per-second of audio.

Buril picks the cloud path by default if a key is configured, falling back to local SDK if you've configured BURIL_AUDIO2FACE_LOCAL_PATH. To force one mode:

voice_audio2face_drive
  audio: <path>
  target: <rig>
  mode: cloud   # or "local"

The output is blendshape curves matching Audio2Face's standard 52-shape ARKit-style topology. If your character's rig uses different shape names, Buril retargets via the voice_a2f_retarget_blendshapes tool.

uLipSync (local, free)

uLipSync is an MIT-licensed Unity package that does viseme-based lipsync at runtime. The source is absorbed under Editor/Tools/Voice/uLipSync/ and ships in 1.0 without a separate install.

• Runs at runtime (not edit-time), so it's the only option for procedural dialogue.
• Maps audio amplitude + formant features to one of 5 visemes (AEIOU + closed-mouth).
• Mobile-friendly: less than 1ms CPU per character per frame.

Configure on a character:

voice_ulipsync_install --character Wizard.HeadRig
voice_ulipsync_bind_audio_source --audio AudioSource_Dialogue

After install, uLipSync drives blendshapes automatically whenever the bound AudioSource plays.

Sentis-local TTS

For fully offline workflows (no ElevenLabs key, no internet), Buril ships a Sentis-based local TTS. Quality is lower than ElevenLabs — fine for prototyping, generally not ship-quality.

voice_sentis_local_generate
  text: "..."
  voice: default

Model weights (~400 MB) download on first call to Library/BurilCache/voice/sentis/.

Lipsync at scale

For projects with hundreds of NPC lines, the production pattern is:

1. Generate all audio with ElevenLabs (offline / pre-production).
2. Bake Audio2Face animation clips for all hero lines.
3. Use uLipSync at runtime for ambient / barked NPC lines that share an AudioSource.

The bridge has a batch tool voice_pipeline_batch that walks a CSV of lines and produces audio + animation per row, with checkpointing so a failed mid-run can resume.

Common issues

Symptom	Fix
Voice sounds robotic	ElevenLabs voice picked too "stable". Set stability: 0.3, style: 0.6 for more variation.
Audio2Face output flat	Audio is too quiet or noisy. Normalize first: audio_normalize_lufs --target -16 LUFS.
uLipSync mouth twitches	Audio source has too much background noise. Add voice_ulipsync_set_threshold to a higher floor.
Sentis model download slow	First-run only. Cached at Library/BurilCache/voice/sentis/.

Read next

• AI animation — body animation (uLipSync handles facial only).
• Multi-Agent Studio — the voice-author specialist.