From decision to acquisition: loss-driven Bayesian active learning

We propose a principled loss-driven approach to Bayesian active learning that allows acquisitions to be directly tailored to downstream decision problems. Whereas current popular information-theoretic approaches implicitly optimise for predictive log loss, our framework generalises this to a very general family of losses comprising of weighted Bregman divergences. We show how choosing any loss from this family defines a unique loss-driven acquisition function by following first principles Bayesian decision theory. We then exploit the fact that Bregman divergences can be analytically minimised to derive a general-purpose estimator for any such resulting acquisition function, allowing it to be used in practice. In classification and regression experiments, we find that our approach can effectively tailor acquisitions, improving performance on different downstream losses.