Research Seminars @ Illinois

Abstract: How does competition affect existential risk in an AGI race? We analyze a model in which firms (or countries) allocate resources between speed of development and safety investment. Firms face random completion times, and the probability of catastrophe depends on the speed-to-safety ratio of whichever firm first reaches AGI. If catastrophe occurs, all firms suffer a large loss; otherwise, the winner captures a prize. Firms thus trade off speed and winning probability against catastrophic risk. We derive a symmetric equilibrium and show that its comparative statics differ from standard winner-take-all contests. More competitors weaken the incentive to race by reducing each firm's marginal impact on winning and shrinking individual resources. However, because the cost of racing faster is only felt conditional on winning, more rivals dilute this private penalty — partially offsetting the first effect. We show that in equilibrium aggregate speed increases and the speed-to-safety ratio falls, speeding up AGI's arrival time and the probability of catastrophe upon arrival. When the number of firms is sufficiently large, individual expected payoffs turn negative. In all cases, firms move faster than is socially optimal. When firms are countries, an additional motive emerges: racing faster to prevent a rival from winning. We discuss interventions by domestic and global social planners, including speed caps and government-owned firms.

This is the joint work of Wioletta Dziuda and Ethan Bueno de Mesquita.