Phase 3: Frontier Governance · ~60 minutes · Python (stdlib · logistic-fit horizon estimator)

METR Time Horizons and External Capability Evaluation

METR (ex-ARC Evals) is an independent 501(c)(3) since December 2023.

Hiring signal: Can operate metr time horizons and external capability evaluation in production

Introduction

Their Time Horizon 1.1 benchmark (January 2026) fits a logistic curve to task-success probability vs log(expert human completion time); the intersection at 50% probability defines the model's time horizon. The 2025–2026 engagement set covers GPT-5.1, GPT-5.1-Codex-Max, and prototype monitoring evaluations (can a monitor catch side tasks; can the agent evade). Benchmark suites: HCAST (180+ ML, cyber, SWE, reasoning tasks; 1 minute to 8+ hours), RE-Bench (71 ML research-engineering tasks with expert baseline), SWAA. The honest note: METR measurements are idealized — no human, no real consequences — and the team has documented the eval-vs-deployment behavior gap (Lesson 1). A time horizon is an upper bound, not a deployment prediction.

Type: Learn Languages: Python (stdlib, logistic-fit horizon estimator) Prerequisites: Phase 15 · 01 (Long-horizon agents), Phase 15 · 19 (RSP) Time: ~60 minutes

Objective

Learning objectives

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.