Why AI Scores Don't Tell You Much Anymore

The tests people run on AI today only show one sliver of what an agent can do, and I think benchmarking has to change. Scott asked in a Super Chat what's state of the art right now and how my bot strategy is going. For me, Opus 5.5 is amazing as a coding work engine and as something I drive day to day. But if your kid is only tested on cooking one meal, you'd never see that she can draw things that make people cry, and AI tests are the same way. What I want to know is whether a model can manage a swarm of agents, keep them running for a long time, update as the world changes, and pick the righ