Phase 2: Models, Serving & Observability · 25 hours · TypeScript (UI) · Python / TypeScript (ingest + evals) · SQL (ClickHouse)

Capstone 11 — LLM Observability & Eval Dashboard

Langfuse went open-core.

Hiring signal: Can build capstone 11 end to end

Introduction

Arize Phoenix published the 2026 GenAI semconv mappings. Helicone and Braintrust both doubled down on per-user cost attribution. Traceloop's OpenLLMetry became the de-facto SDK instrumentation. The production shape is ClickHouse for traces, Postgres for metadata, Next.js for UI, and a small army of eval jobs (DeepEval, RAGAS, LLM-judge) running over sampled traces. Build one self-hosted, ingest from at least four SDK families, and demonstrate catching an injected regression in under five minutes.

Phases exercised: P11 · P13 · P17 · P18

Type: Capstone Languages: TypeScript (UI), Python / TypeScript (ingest + evals), SQL (ClickHouse) Prerequisites: Phase 11 (LLM engineering), Phase 13 (tools), Phase 17 (infrastructure), Phase 18 (safety) Time: 25 hours

Objective

Learning objectives

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.