Phase 6: Real-Time Transport & Telephony · 55 min · Python · aiortc · LiveKit SDK

WebRTC Architecture: SDP, ICE, DTLS, SRTP

WebRTC saves 700ms over phone calls. Understanding the protocol stack tells you why.

Real-world application: WebRTC architecture knowledge is essential for real-time systems engineer (voice) roles.

What you will learn

Understand the WebRTC protocol stack: SDP, ICE, DTLS, SRTP
Explain why WebRTC saves ~700ms over phone calls
Implement WebRTC audio connections with aiortc
Configure STUN/TURN servers for NAT traversal

The Problem

Your voice agent needs to receive audio from the caller's browser and send audio back — in real time, with sub-500ms latency. HTTP is too slow (request-response). You need a persistent, bidirectional, low-latency channel. WebRTC is the standard for browser-based real-time audio.

The Concept

The WebRTC Protocol Stack

Layer	Protocol	Purpose
Signaling	SDP over WebSocket	Negotiate codecs, formats, capabilities
Connection	ICE/STUN/TURN	Establish peer-to-peer connection through NAT
Security	DTLS	Encrypt the connection
Media	SRTP	Secure Real-time Transport Protocol — audio packets
Data	SCTP over DTLS	Non-media data (optional)

Why does WebRTC use ICE (Interactive Connectivity Establishment) instead of just connecting directly?

Most devices are behind NAT (routers, firewalls) — they can't receive direct connections. ICE solves this by trying multiple candidate paths: (1) Host candidates — direct local IP, (2) Server-reflexive candidates — public IP discovered via STUN, (3) Relay candidates — traffic relayed through TURN server. ICE tries each in order and picks the best working path. Without ICE, WebRTC connections would fail for anyone behind a NAT, which is ~90% of users. STUN discovers the public address; TURN is the fallback relay when P2P fails.

SDP (Session Description Protocol)

SDP is the negotiation format — both sides describe what they can do:

# SDP Offer (browser → server)
sdp_offer = """
v=0
o=- 45789 1 IN IP4 203.0.113.1
s=-
t=0 0
m=audio 49170 RTP/SAVPF 111
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1
a=sendrecv
"""

# SDP Answer (server → browser)
sdp_answer = """
v=0
o=- 89201 1 IN IP4 198.51.100.1
s=-
t=0 0
m=audio 49172 RTP/SAVPF 111
a=rtpmap:111 opus/48000/2
a=sendrecv
"""

ICE Candidate Types

Type	How	Speed	Reliability
Host	Direct local IP	Fastest	Fails behind NAT
Server-reflexive (srflx)	STUN-discovered public IP	Fast	Works through NAT
Relay	TURN server relays traffic	Slowest	Always works

Audio Codecs in WebRTC

Codec	Bitrate	Quality	Latency	Mandatory?
Opus	6-510 kbps	Excellent	~26.5ms	Yes (required)
G.711 (PCMU/PCMA)	64 kbps	Fair	~20ms	Yes (legacy)
G.722	64 kbps	Good	~20ms	No
iLBC	15.2 kbps	Fair	~30ms	Deprecated

Opus is the codec for modern voice agents. It handles both narrowband (8kHz) and full-band (48kHz) audio, adapts bitrate to network conditions, and has built-in FEC (Forward Error Correction).

WebRTC for Voice Agents

# Server-side WebRTC handling (Python with aiortc)
from aiortc import RTCPeerConnection, RTCSessionDescription

async def handle_webrtc_offer(sdp_offer):
    """Handle incoming WebRTC connection from browser."""
    pc = RTCPeerConnection()

    @pc.on("track")
    def on_track(track):
        """Receive audio track from browser."""
        if track.kind == "audio":
            # Feed audio to ASR pipeline
            asyncio.create_task(process_audio(track))

    # Set remote description (browser's offer)
    await pc.setRemoteDescription(RTCSessionDescription(
        sdp=sdp_offer, type="offer"
    ))

    # Create and send answer
    answer = await pc.createAnswer()
    await pc.setLocalDescription(answer)
    return answer.sdp

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.