The Concept
Emotional AI: Hume EVI
Hume's EVI analyzes prosodic features (pitch, energy, rhythm, voice quality) to detect emotions in real-time:
| Emotion | Signal | Agent Adaptation |
|---|
| Frustration | High pitch, fast pace | Slow down, empathize |
| Sadness | Low pitch, slow pace | Gentle tone, patience |
| Joy | High energy, varied pitch | Match enthusiasm |
| Anxiety | Rapid pace, breathy | Reassure, simplify |
| Anger | Loud, sharp consonants | De-escalate, don't argue |
| Confusion | Hesitation, rising intonation | Clarify, simplify |
How does Hume EVI detect emotion differently from sentiment analysis on transcripts?
Hume EVI analyzes prosodic features (pitch, energy, rhythm, voice quality) directly from audio — detecting emotion from HOW something is said, not WHAT is said. Transcript sentiment analysis only sees words, missing sarcasm ("oh GREAT"), anger masked by polite words, or distress in a calm voice
Hume EVI Integration
class HumeEmotionDetector:
def __init__(self):
self.hume_client = HumeClient(api_key="your-key")
async def detect_emotions(self, audio_chunk):
"""Detect emotions from audio in real-time."""
result = await self.hume_client.predict(audio_chunk)
return {
"emotions": result["emotions"], # [{"name": "frustration", "score": 0.82}, ...]
"prosody": result["prosody"], # {"pitch": "high", "energy": "tense", "rate": "fast"}
"top_emotion": result["emotions"][0]["name"],
"confidence": result["emotions"][0]["score"],
}
def adapt_response(self, emotion, confidence):
"""Adapt agent response based on detected emotion."""
adaptations = {
"frustration": "I understand this is frustrating. Let me help you right away.",
"sadness": "I'm here for you. Take your time.",
"anxiety": "Don't worry, I'll guide you through this step by step.",
"anger": "I hear your concern. Let me see what I can do immediately.",
"confusion": "Let me explain that more clearly.",
"joy": "That's wonderful! I'm happy to help with that.",
}
return adaptations.get(emotion, "How can I help?")
Emotion-Adaptive Agent
class EmotionAdaptiveAgent:
def __init__(self):
self.emotion_detector = HumeEmotionDetector()
self.emotion_history = []
async def respond(self, user_audio, user_text):
"""Generate emotion-aware response."""
# Detect emotion from audio
emotion_data = await self.emotion_detector.detect_emotions(user_audio)
self.emotion_history.append(emotion_data)
# Adapt system prompt based on emotion
adapted_prompt = self._build_adaptive_prompt(emotion_data)
# Generate response with emotion context
response = await self.llm.generate(
user_text,
system_prompt=adapted_prompt,
emotion_context=emotion_data["top_emotion"],
)
# Adjust TTS voice parameters
tts_params = self._adjust_tts(emotion_data["top_emotion"])
return {
"text": response,
"emotion_detected": emotion_data["top_emotion"],
"tts_params": tts_params,
}
def _build_adaptive_prompt(self, emotion_data):
"""Build system prompt adapted to caller's emotion."""
emotion = emotion_data["top_emotion"]
base_prompt = "You are a helpful voice agent."
emotion_prompts = {
"frustration": f"{base_prompt} The caller is frustrated. Be empathetic, concise, and focus on immediate solutions.",
"sadness": f"{base_prompt} The caller seems sad. Use a gentle, patient tone.",
"anxiety": f"{base_prompt} The caller seems anxious. Be reassuring and break things into simple steps.",
"anger": f"{base_prompt} The caller is angry. Don't argue. Acknowledge their concern and act quickly.",
}
return emotion_prompts.get(emotion, base_prompt)
def _adjust_tts(self, emotion):
"""Adjust TTS parameters based on emotion."""
adjustments = {
"frustration": {"speed": 0.9, "pitch": -2}, # Slower, lower
"sadness": {"speed": 0.85, "pitch": -3}, # Slowest, lowest
"anxiety": {"speed": 0.95, "pitch": 0}, # Slightly slower
"anger": {"speed": 0.9, "pitch": -1}, # Calm, measured
"joy": {"speed": 1.05, "pitch": 2}, # Slightly faster, higher
}
return adjustments.get(emotion, {"speed": 1.0, "pitch": 0})
Voice Biometrics
Voice biometrics verifies identity from vocal characteristics:
| Feature | What It Measures | Uniqueness |
|---|
| Pitch (F0) | Fundamental frequency | Gender, age |
| Formant frequencies | Vocal tract shape | Individual anatomy |
| Spectral envelope | Overall voice timbre | Highly unique |
| Speech rate | Words per minute | Habitual pattern |
| Pronunciation patterns | Articulation habits | Regional + individual |
| Voice quality | Breathiness, roughness | Individual trait |
class VoiceBiometricSystem:
def __init__(self):
self.voiceprints = {} # user_id → voiceprint embedding
self.threshold = 0.85 # Verification threshold
async def enroll(self, user_id, audio_samples):
"""Enroll a user's voiceprint."""
# Extract voice features from multiple samples
embedding = await self._extract_features(audio_samples)
self.voiceprints[user_id] = embedding
return {"user_id": user_id, "enrolled": True}
async def verify(self, user_id, audio_sample):
"""Verify a user's identity from voice."""
if user_id not in self.voiceprints:
return {"verified": False, "reason": "not_enrolled"}
# Extract features from sample
embedding = await self._extract_features([audio_sample])
# Compare with stored voiceprint
similarity = self._cosine_similarity(self.voiceprints[user_id], embedding)
return {
"verified": similarity >= self.threshold,
"similarity": similarity,
"threshold": self.threshold,
}
Voice Biometrics vs Traditional Auth
| Factor | Voice Biometrics | PIN/Password | OTP |
|---|
| User effort | None (passive) | Must remember | Must receive + enter |
| Security | Hard to fake voice | Can be shared | Can be intercepted |
| Friction | Zero (natural speech) | High | Medium |
| Accessibility | Good for visually impaired | Difficult | Difficult |
| Cost | Per-verification fee | Free | SMS costs |
| False accept rate | ~0.01% | Depends on PIN length | N/A |
| False reject rate | ~2-5% | Low | Low |
| Liveness detection | Yes (anti-spoofing) | No | No |
Anti-Spoofing
| Attack Type | How It Works | Defense |
|---|
| Replay attack | Play recorded voice | Liveness detection (random phrase) |
| Voice synthesis | TTS with cloned voice | Synthetic voice detection |
| Voice conversion | Transform attacker's voice | Spectral artifact detection |
| Deepfake audio | AI-generated voice | Watermarking + detection models |