Metrics of Agent Ability

**Goals of this post** > > **Goal:** a simple way to compare a variety of capability metrics in an idealized setting where we observe only a function from $x$ to $s$. > > Non-goals: > > 1. Treatment of practical difficulties (an incomplete list of other considerations appears at the bottom). > 2. Recommendations about which metric to use. > 3. Representation of a consensus METR view. {: .metrics-callout} --> This post reviews alternative metrics of agent capability. This post contains a fairly comprehensive taxonomy of capability metrics for the situation in which you observe score curves for