On 3rd May 2022, the British and Irish Region of the Biometric Society hosted an online meeting on Variational Bayes. Variational Bayes is a computational method that allows scalable Bayesian inference that can be used as an alternative to the more commonly used Markov Chain Monte Carlo methods.
The meeting consisted of an introduction to Variational Bayes given in the morning by Dr David Hughes (University of Liverpool). David introduced the basic concepts of variational approximations using a simple example to demonstrate the main ideas. Mean Field Variational Bayes attempts to approximate a complex posterior using a product of simpler distributions. Although the approximation can lead to slightly less accurate posterior estimates than would be obtained by MCMC, the algorithmic (rather than sampling) nature of VB methods, using distributions with nice, known properties, can significantly improve computational time. By choosing an approximate distribution that is in some sense “close” to the desired posterior (commonly defined using a Kullback-Leibler divergence) the approximate solutions is usually quite close to the desired complicated posterior. The session was attended by over 70 participants and led to many questions and good discussion following the talk.
The afternoon consisted of three talks, each describing different areas of state-of-the-art use of variational Bayes methods. The session was attended by around 60 people, and again each talk prompted good discussion. Dr Hélène Ruffieux (University of Cambridge) and winner of the 2021 Young Biometrician of the Year award, gave the first talk in which she described the use of variational Bayes methods in genome wide association studies. Hélène showed that standard variational algorithms can lead to inaccuarate results, but by the use of a simulated annealing procedure within the variational algorithm, much more accurate results can be obtained with very little additional computational complexity. Hélène then demonstrated how this approach can allow investigation of associations between genetic SNPs and various Omics type data, an even more high-dimensional setting. Her results both confirmed existing known genetic hotspots, and identified potential others in models that what be computationally expensive, or even prohibitive using MCMC approaches.
Dr Kamélia Daudel (University of Oxford) gave the second talk of the afternoon. Her work was more theoretical in nature and explored alternative ways of measuring the “closeness” of the variational approximation to the desired posterior. Her proposal was to consider alpha-divergence and showed that this was better able to estimate some more complex posterior distributions (such as mixture distributions). Kamélia described two algorithms for performing variational inference using monotone alpha-divergence measures, that allow mixture weights and mixture component parameters to be updated simultaneously. She also showed the theoretical guarantees possible with her approach, and showed through some numerical studies the benefits of the method.
The final talk of the afternoon was given by Dr Tui Nolan (University of Cambridge). Tui’s work focused on a technique called variational message passing. This is an alternative way of arranging the variational Bayes algebra, that links to DAG models and is a simple way of building up the necessary parts of a variational algorithm, by building on the existing work of others, and only deriving the additional extra “fragments” required for the desired model. Tui described his work on functional principal components analysis for rainfall data from the USA, collected over many years. Tui’s models involved a multilevel model and identified principal components that describe differences in rainfall between states, and within states over time. His variational algorithm allowed complex multi-level models to be estimated in a computationally efficient manner.
The meeting was well received and provide both a general overview of variational inference methods, and three novel and complex uses of the methodology. Recordings of the sessions, and the slides for each talk are available to BIR members on the Biometric society website (https://bir.biometricsociety.org/bir/viewdocument/variational-bayes-meeting?CommunityKey=431dc917-fff5-4514-90c0-a56772737e82).