AI4OPT Seminar Series

Date: Thursday, Aug 31, 2023

Time: Noon – 1:00 pm

Location: J. Erskine Love Jr. Manufacturing Building - Classroom 183 (771 Ferst Dr NW Atlanta, GA 30318)

Join Virtual: https://gatech.zoom.us/j/99381428980

Speaker: Anqi Wu

Improving Variational Inference for Complex Probabilistic Modeling

Abstract: In the Bayesian setting, the primary objective is to learn the true posterior distribution for model parameters given the observations and prior distributions. The central hurdle revolves around integrating out all variables within the model, excluding the variable of interest. This intricate high-dimensional integration typically poses substantial computational challenges, which has prompted the necessity for approximate inference techniques to solve such intricate probabilistic inference problems. One of the well-known methods for approximate inference in probabilistic modeling is variational inference (VI). Variational inference deals with finding an approximate posterior distribution for Bayesian models where finding the true posterior distribution is analytically or numerically impossible. It assumes a family of distribution for the estimation, and finds the closest member of that family to the true posterior distribution using a distance measure. VI can be used to solve many complicated probabilistic modeling problems. Nevertheless, traditional variational inference approaches are not without their limitations. It is within this context that I introduce two groundbreaking variations of variational inference algorithms. In the first part, I will unveil a deterministic approach to variational inference designed specifically for Bayesian neural networks (BNNs). This innovative technique directly addresses two pivotal challenges encountered in the conventional variational inference for BNNs: successful implementations require careful initialization and tuning of prior variances, as well as controlling the variance of Monte Carlo gradient estimates. The resultant methodology emerges as remarkably efficient and robust. Transitioning to the second part, I will introduce a novel approach termed "variational importance sampling" (VIS) designed for latent variable models. Our new methodology ingeniously combines the principles of importance sampling and variational inference. It yields a substantially tighter lower-bound to the log marginal compared to the widely used Evidence Lower Bound (ELBO). This innovation is brought to bear upon the challenging problem of partially observed multivariate Hawkes processes (POMHP), the effective inference of which has been generally under-explored. We shall demonstrate how VIS provides an effective and efficient means of inference to address the intricacies of this model. Empirical results underscore its superiority, as evidenced by performance on synthetic datasets and, notably, across four real-world neural datasets. These results also serve to emphasize the practical utility of POMHP with VIS in deciphering neural connectivity from recorded spike trains.

Bio: Anqi Wu is an Assistant Professor at the School of Computational Science and Engineering (CSE), Georgia Institute of Technology. She was a Postdoctoral Research Fellow at the Center for Theoretical Neuroscience, the Zuckerman Mind Brain Behavior Institute, Columbia University. She received her Ph.D. degree in Computational and Quantitative Neuroscience and a graduate certificate in Statistics and Machine Learning from Princeton University. Anqi was selected for the MIT Rising Star in EECS, DARPA Riser, and Alfred P. Sloan Fellow. Her research interest is to develop scientifically-motivated Bayesian statistical models to characterize structure in neural data and behavior data in the interdisciplinary field of machine learning and computational neuroscience. She has a general interest in building data-driven models to promote both animal and human studies in the system and cognitive neuroscience.

Lunch will be served at the seminar. So, please stop by 15 minutes before the seminar to pick up lunch.

To receive AI4OPT seminar announcements, please sign up to our mailing list. at https://lists.isye.gatech.edu/mailman/listinfo/ai4opt-seminars.

Past seminars can be found at https://www.ai4opt.org/seminars/past-seminars.