Third Moment Research
ML Systems & Data Research
The Third Moment
μ = E[X]
First moment
The center: the average eval score. Most teams hill‑climb this number, and it can rise while real failures stay put.
σ² = E[(X−μ)²]
Second moment
The spread: how much the score moves from run to run. A gain smaller than the noise is not a gain.
μ₃ = E[(X−μ)³]
Third moment
The tail: the rare, hard cases an average smooths over. We build evals that target them. This is where we climb.