Title: An Introduction to Stochastic Deep Learning
Abstract: We have developed a new type of stochastic neural network (StoNet), which is formulated as a composition of many simple linear/logistic regression models, and designed an adaptive stochastic gradient MCMC algorithm for its training. The StoNet fits into the framework of statistical modeling, allowing us not only to address fundamental issues in deep learning, such as structural interpretability and uncertainty quantification, but also to provide a platform for transferring the theory and methods developed for linear models to deep learning. We showcase the integration of reproducing kernel methods into deep neural networks to enhance their training and prediction performance. Furthermore, we demonstrate how to use the StoNet to perform nonlinear sufficient dimension reduction and causal inference on high-dimensional data. Lastly, we illustrate how to leverage the StoNet to handle special types of data, such as those with missing values or measurement errors, and how to use it to perform statistical inference for conventional deep neural networks. This talk is based on joint works with Yan Sun, Siqi Liang, and Yaxin Fang.