Next Generation Algorithms for Stochastic Optimization with Constraints
Abstract: Stochastic gradient and related methods for solving stochastic optimization problems have been studied extensively in recent years. It has been shown that such algorithms and much of their convergence and complexity guarantees extend in straightforward ways when one considers problems involving simple constraints, such as when one can perform projections onto the feasible region of the problem. However, settings with general nonlinear constraints have received less attention, and many of the approaches that have been proposed for solving such problems resort to using penalty or (augmented) Lagrangian methods, which are often not the most effective strategies. In this work, we propose and analyze stochastic optimization algorithms for deterministically constrained problems based on the sequential quadratic optimization (commonly known as SQP) methodology. We discuss the rationale behind our proposed techniques, convergence in expectation and complexity guarantees for our algorithms, and present numerical experiments that we have performed. This is joint work with Raghu Bollapragada, Frank E. Curtis, Wanping Dong, Michael O'Neill, Daniel P. Robinson, Jiahao Shi and Baoyu Zhou.
Biography: Albert S. Berahas is an Assistant Professor in the Industrial and Operations Engineering department at the University of Michigan. Before joining the University of Michigan, he was a Postdoctoral Research Fellow in the Industrial and Systems Engineering department at Lehigh University (working with Professors Katya Scheinberg, Frank E. Curtis and Martin Takáč), and prior to that appointment, he was a Postdoctoral Research Fellow in the Industrial Engineering and Management Sciences department at Northwestern University (working with Professor Jorge Nocedal). Berahas completed his PhD studies in the Engineering Sciences and Applied Mathematics (ESAM) department at Northwestern University in 2018, advised by Professor Jorge Nocedal. He received his undergraduate degree in Operations Research and Industrial Engineering (ORIE) from Cornell University in 2009, and in 2012 obtained an MS degree in Applied Mathematics from Northwestern University. Berahas’ research broadly focuses on designing, developing and analyzing algorithms for solving large scale nonlinear optimization problems. Specifically, he is interested in and has explored several sub-fields of nonlinear optimization such as: (i) constrained optimization, (ii) optimization algorithms for machine learning, (iii) stochastic optimization, (iv) derivative-free optimization, and (v) distributed optimization. Berahas served as the vice-chair of the Nonlinear Optimization cluster for the INFORMS Optimization Society (2020-2022), the chair of the Nonlinear Optimization cluster for the INFORMS Optimization Society Conference (2021-2022), the co-chair of the Nonlinear Optimization cluster for the ICCOPT 2022 conference (2021-2022), and served as the president of the INFORMS Junior Faculty Interest Group (JFIG) from 2023-2024. Berahas was awarded the 2022 Charles Broyden Prize, the 2024 IISE Excellence in Teaching of Operations Research Award, the INFORMS Moving Spirit Award for Forums in 2024, and the Martin Luther King, Jr Spirit Award for Community Building & Impact in 2024.