Abstract: Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale -- 1.6~billion~km of driving. This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour on a single 8-GPU node. The resulting policy achieves state-of-the-art performance on three independent autonomous driving benchmarks. The policy outperforms the prior state of the art when tested on recorded real-world scenarios, amidst human drivers, without ever seeing human data during training. The policy is realistic when assessed against human references and achieves unprecedented robustness, averaging 17.5 years of continuous driving between incidents in simulation. We will then discuss some ongoing work building atop these results investigating relationships between data scale and robustness in mixed human-autonomous systems.
Bio: Eugene Vinitsky is an assistant professor of civil and urban engineering at NYU where he works on scaling up multi-agent reinforcement learning for the design of safe, autonomous systems. At UC Berkeley, where he was advised by Alexandre Bayen, he received his PhD in controls engineering with a specialization in reinforcement learning and received an MS and BS in physics from UC Santa Barbara and Caltech respectively. He has spent time at Tesla, Deepmind, Facebook AI Research, and was a researcher at the Apple Special Project Group before moving to NYU.
