From Decision to Acquisition: Loss-Driven Bayesian Active Learning

Bayesian active learning has become practically synonymous with maximising expected information gain (EIG) during data acquisition. We highlight that this standard practice makes an implicit assumption about the underlying decision problem of interest, with this assumption not reflecting our goals in many cases. Generalising EIG maximisation, we propose an explicitly loss-driven approach to Bayesian active learning, with which we can target reduced loss in a much broader class of decision problems. We also identify a large family of losses for which we can derive efficient estimators of our principled data-acquisition objective. In practical classification and regression demonstrations, we find our approach can produce a notable performance boost over existing techniques.