Optimization Over Trained Neural Networks: Taking a Relaxing Walk
Thiago Serra
Assistant Professor, Tippie College of Business
University of Iowa
Abstract: Besides training, mathematical optimization is also used in deep learning to model and solve formulations over trained neural networks for purposes such as verification, compression, and optimization with learned constraints. However, solving these formulations soon becomes difficult as the network size grows due to the weak linear relaxation and dense constraint matrix. We have seen improvements in recent years with cutting plane algorithms, reformulations, and an heuristic based on Mixed-Integer Linear Programming (MILP). In this work, we propose a more scalable heuristic based on exploring global and local linear relaxations of the neural network model. Our heuristic is competitive with a state-of-the-art MILP solver and the prior heuristic while producing better solutions with increases in input, depth, and number of neurons.
Biography: Thiago Serra recently joined the University of Iowa's Tippie College Business as Assistant Professor of Business Analytics, following 5 years as Assistant Professor of Analytics and Operations Management at Bucknell University's Freeman College of Management. Previously, he was Visiting Research Scientist at Mitsubishi Electric Research Labs from 2018 to 2019, and Operations Research Analyst at Petrobras from 2009 to 2013. He has a Ph.D. in Operations Research from Carnegie Mellon University's Tepper School of Business, from which he received the Gerald L. Thompson Doctoral Dissertation Award in Management Science in 2018. During his Ph.D., he was also awarded the INFORMS Judith Liebman Award. His research at the intersection of discrete optimization and machine learning is supported by the National Science Foundation.