Research Seminars @ Illinois

Title: Ergodic risk-sensitive control of diffusions under a general structural hypothesis and computational and learning algorithms.

Abstract: In this talk, we will first introduce a new theoretical framework for a broad class of ergodic risk-sensitive diffusion control problems. In particular, we develop the theory under a general structural hypothesis that comprises the existing theoretical frameworks under either uniform stability or near-monotone conditions as special cases. This is necessary for the ergodic risk-sensitive control of the limiting diffusion for large-scale parallel server networks. By employing the variational formulas for the exponential functionals of Brownian motion, we exploit the representation as a two-person zero-sum stochastic differential game under the ergodic criterion, for the associated extended diffusion. We will next discuss some relevant computational and learning algorithms for ergodic risk-sensitive optimal control problems. We start with new Jacobi-like and Gauss-Seidel-like relative value iteration (RVI) algorithms for the ergodic risk-sensitive control problems for Markov chains. We then present the entropy-based risk-sensitive Q-learning algorithms corresponding to the existing and new Jacobi-like RVI algorithms. We finally present RVI algorithms for ergodic risk-sensitive control of diffusions and their rate of convergence. In the end, we will discuss some future research directions and open problems.