|
The Colloquium on Digital Transformation is a series of weekly online talks on how artificial intelligence, machine learning, and big data can lead to scientific breakthroughs with large-scale societal benefit. The spring 2021 series focuses largely on COVID-19 mitigation research.
See details of upcoming talks here and note we have the same
Zoom Webinar registration link for all forthcoming talks
Recent Advances in Analysis of Implicit Bias of Gradient Descent on Deep Networks
April 8, 3 pm CST Matus Telgarsky, Assistant Professor of Computer Science
University of Illinois at Urbana-Champaign
|
ABSTRACT: The purpose of this talk is to highlight three recent directions in the study of implicit bias, a promising approach to developing a tight generalization theory for deep networks interwoven with optimization. The first direction is a warm-up with purely linear predictors: here, the implicit bias perspective gives the fastest known hard-margin SVM solver! The second direction is on the early training phase with shallow networks: here, implicit bias leads to good training and testing error, with not just narrow networks but also arbitrarily large ones. The talk concludes with deep networks, providing a variety of structural lemmas that capture foundational aspects of how weights evolve for any width and sufficiently large amounts of training. This is joint work with Ziwei Ji. Matus Telgarsky is an Assistant Professor of Computer Science at the University of Illinois at Urbana-Champaign, specializing in deep learning theory. He received a PhD at the University of California, San Diego under Sanjoy Dasgupta. He co-founded the Midwest ML Symposium in 2017 with Po-Ling Loh and organized a Simons Institute summer 2019 program on deep learning with Samy Bengio, Aleskander Madry, and Elchanan Mossel. He received an NSF CAREER Award in 2018.
|
|
|
|
