Third Moment Research

ML Systems & Data Research

μ = E[X]

First moment

The center: the average eval score. Most teams hill‑climb this number, and it can rise while real failures stay put.

σ² = E[(X−μ)²]

Second moment

The spread: how much the score moves from run to run. A gain smaller than the noise is not a gain.

μ₃ = E[(X−μ)³]

Third moment

The tail: the rare, hard cases an average smooths over. We build evals that target them. This is where we climb.