Skip to yearly menu bar Skip to main content


Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support

Tim Reichelt · Luke Ong · Tom Rainforth

MR1 & MR2 - Number 153
[ ]
Thu 2 May 8 a.m. PDT — 8:30 a.m. PDT


The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path.We show that making predictions with this full posterior implicitly performs a Bayesian model averaging (BMA) over paths.This is potentially problematic, as BMA weights can be unstable due to model misspecification or inference approximations, leading to sub-optimal predictions in turn.To remedy this issue, we propose alternative mechanisms for path weighting: one based on stacking and one based on ideas from PAC-Bayes.We show how both can be implemented as a cheap post-processing step on top of existing inference engines.In our experiments, we find them to be more robust and lead to better predictions compared to the default BMA weights.

Live content is unavailable. Log in and register to view live content