The simple reason AI can’t bootstrap superintelligence

AI systems are starting to build themselves. The standard story is that recursive self-improvement will lead to superintelligence. Host Tom Reed does not think this will happen. Nothing — not even a country of geniuses — gets good at solving problems without a supply of real problems. For most tasks in the economy, the data required doesn't exist. It only comes into being when a model is deployed and somebody reveals a preference. An isolated AI improving itself against benchmarks would only _appear_ to be approaching superintelligence, while actually optimising for eval performance that fail