Momentum SVGD-EM for Accelerated Maximum Marginal Likelihood Estimation

Maximum marginal likelihood estimation (MMLE) can be recast as the optimization of the so-called free energy. The celebrated expectation-maximisation (EM) procedure can then be interpreted as a coordinate-descent scheme in the space of parameters and measures. Recently, a significant body of work has adopted this perspective, leading to interacting particle algorithms for MMLE. In this paper, we propose an accelerated version of one such procedure, based on Stein variational gradient descent (SVGD), by introducing Nesterov momentum in both the parameter updates and in the space of probability measures. The resulting method, termed Momentum SVGD-EM, consistently accelerates convergence in terms of required iterations across various tasks of increasing difficulty, demonstrating effectiveness in both low- and high-dimensional settings.