We look forward to seeing you in person or online on Monday, November 14, at 4:30pm. Join in person at 2124 Siebel Center for Computer Science, 201 N. Goodwin Ave or vitually on Zoom, https://illinois.zoom.us/j/89400978467?pwd=NjV3ZFQrQ1JidTNyS0ZNUEVOcEtpUT09
Abstract:
We present a novel, general construction to abstractly interpret higher-order automatic differentiation (AD). Our construction allows one to instantiate an abstract interpreter for computing derivatives up to a chosen order. Furthermore, since our construction reduces the problem of abstractly reasoning about derivatives to abstractly reasoning about real-valued straight-line programs, it can be instantiated with almost any numerical abstract domain, both relational and non-relational. We formally establish the soundness of this construction.
We implement our technique by instantiating our construction with both the non-relational interval domain and the relational zonotope domain to compute both first and higher-order derivatives. In the latter case, we are the first to apply a relational domain to automatic differentiation for abstracting higher-order derivatives, and hence we are also the first abstract interpretation work to track correlations across not only different variables, but different orders of derivatives.
We evaluate these instantiations on multiple case studies, namely robustly explaining a neural network and more precisely computing a neural​ network’s Lipschitz constant. For robust interpretation, first and second derivatives computed via zonotope AD are up to 4.76× and 6.98× more precise, respectively, compared to interval AD. For Lipschitz certification, we obtain bounds that are up to 11,850× more precise with zonotopes, compared to the state-of-the-art interval-based tool.
Bio:
Jacob Laurel is a 6th year PhD Student at the University of Illinois Urbana-Champaign advised by Prof. Sasa Misailovic. Jacob's current research focuses on leveraging programming languages to improve automated reasoning or new Machine Learning-driven domains and computational paradigms including differentiable and probabilistic programming. His research has been presented at top-tier Programming Languages, Embedded Systems and machine Learning conferences and has been generously supported through a UCEM PhD fellowship from the Alfred P. Sloan Foundation.
