Bringing Statistical Thinking in Decentralized Optimization. Vignettes from High-dimensional Statistics over Networks.
Gesualdo Scutari
Professor, School of Engineering, Purdue University
Abstract: There is growing interest in solving large-scale statistical learning problems over decentralized networks, where data are distributed across the nodes of the network and no centralized coordination is present (we termed these systems as “mesh” networks). Inference from massive datasets poses a fundamental challenge at the nexus of the computational and statistical sciences: ensuring the quality of the performed analytic when computational resources, like time and communication, are constrained. While statistical-computation tradeoffs have been largely explored in the centralized setting, our understanding over mesh networks is limited: (i) distributed schemes, designed and performing well in the classical low-dimensional regime, can break down in the high-dimensional case; and (ii) existing convergence studies may fail to predict algorithmic behaviors, with some findings directly contradicted by empirical tests. This is mainly due to the fact that the majority of decentralized algorithms have been designed and studied only from the optimization perspective, lacking the statistical dimension.
This talk will discuss some vignettes from high-dimensional statistical inference suggesting new analyses and designs aiming at bringing statistical thinking in decentralized optimization, thereby enhancing algorithmic performance and reliability in high-dimensional, decentralized settings. In conclusion, and if time permits, the discussion will extend to the interdisciplinary effort triggered by this vision, leading to the establishment of the Center of Science of Data Analytics (SODA) at Purdue University. This initiative epitomizes the fusion of the triad optimization, data analytics, and networking, aiming at addressing the multifaceted challenges at the heart of decentralized statistical learning and decision making.
Biography: Gesualdo Scutari is a Professor with the School of Industrial Engineering and Electrical and Computer Engineering at Purdue University, West Lafayette, IN, USA, and he is a Purdue Faculty Scholar. His research interests include continuous (distributed, stochastic) optimization, equilibrium programming, and their applications to signal processing and statistical learning. Among others, he was a recipient of the 2013 NSF CAREER Award, the 2015 IEEE Signal Processing Society Young Author Best Paper Award, and the 2020 IEEE Signal Processing Society Best Paper Award. He serves as an IEEE Signal Processing Distinguish Lecturer (2023-2024). He served on the editorial broad of several IEEE journals and he is currently an Associate Editor of SIAM Journal on Optimization. He is a fellow of IEEE.