The Pipeline
Text Script → ElevenLabs TTS → Audio (.mp3)
↓
Portrait Image → OmniHuman 1.5 → Lip-synced Video (.mp4)
↓
Final Output
Step 1: Portrait Image
You need a high-quality portrait photo of the person who will "speak":
# Option A: Use an existing photo
portrait = "spokesperson.png"
# Option B: Generate with FLUX
from fal_client import submit
result = submit("fal-ai/flux/schnell", {
"prompt": "professional headshot of a woman in business attire, neutral background, looking at camera, sharp focus",
"image_size": {"width": 1024, "height": 1024},
})
portrait_url = result["images"][0]["url"]
Portrait Quality Requirements
| Requirement | Why |
|---|
| Front-facing | OmniHuman needs to see the face clearly |
| Good lighting | Poor lighting degrades lip-sync quality |
| Neutral expression | Extreme expressions cause artifacts |
| High resolution | 1024x1024 minimum |
| Single person | Multi-person portraits don't work well |
| Clear mouth | Mouth area must be visible and unobstructed |
Step 2: TTS with ElevenLabs
Generate natural-sounding speech from text:
import requests
ELEVENLABS_API_KEY = os.getenv("ELEVENLABS_API_KEY")
ELEVENLABS_API = "https://api.elevenlabs.io/v1"
def generate_tts(text: str, voice_id: str = "21m00Tcm4TlvDq8ikWAM",
model: str = "eleven_turbo_v2") -> bytes:
"""Generate TTS audio with ElevenLabs."""
response = requests.post(
f"{ELEVENLABS_API}/text-to-speech/{voice_id}",
headers={
"xi-api-key": ELEVENLABS_API_KEY,
"Content-Type": "application/json",
},
json={
"text": text,
"model_id": model,
"voice_settings": {
"stability": 0.5,
"similarity_boost": 0.75,
"style": 0.0,
"use_speaker_boost": True,
},
},
)
return response.content # Audio bytes (MP3)
ElevenLabs Voice Selection
| Voice Type | Use Case | Cost |
|---|
| Pre-made voices | Quick start, testing | $0.30/1K chars |
| Cloned voices | Brand consistency | $0.30/1K chars + cloning |
| Custom voices | Exact match to person | Professional tier |
# List available voices
voices = requests.get(
f"{ELEVENLABS_API}/voices",
headers={"xi-api-key": ELEVENLABS_API_KEY},
).json()
for voice in voices["voices"]:
print(f"{voice['voice_id']}: {voice['name']} ({voice['labels'].get('gender', 'N/A')})")
What is the correct pipeline order for talking head video generation?
The correct order is: (1) Generate TTS audio from the text script using ElevenLabs, (2) Provide the portrait image + audio to OmniHuman, which generates a lip-synced video where the person's mouth movements match the audio. OmniHuman needs the audio first to know what mouth movements to generate.
Step 3: OmniHuman Video Generation
# OmniHuman via fal.ai
response = requests.post(
"https://fal.run/fal-ai/omnihuman",
headers={"Authorization": f"Key {FAL_KEY}"},
json={
"image_url": portrait_url,
"audio_url": audio_url,
"motion_scale": 1.0, # 0.5 = subtle, 1.0 = natural, 1.5 = expressive
}
)
video_url = response.json()["video"]["url"]
OmniHuman Parameters
| Parameter | Range | Effect |
|---|
motion_scale | 0.5-1.5 | 0.5 = subtle motion, 1.0 = natural, 1.5 = expressive |
image_url | URL | Portrait image (front-facing, good lighting) |
audio_url | URL | TTS audio (MP3/WAV from ElevenLabs) |