ExactBoost: Directly Boosting the Margin in Combinatorial and Non-decomposable Metrics

Daniel Csillag · Carolina Piazza · Thiago Ramos · João Vitor Romano · Roberto Oliveira · Paulo Orenstein

[ Abstract ]
Mon 28 Mar 4:30 a.m. PDT — 6 a.m. PDT


Many classification algorithms require the use of surrogate losses when the intended loss function is combinatorial or non-decomposable. This paper introduces a fast and exact stagewise optimization algorithm, dubbed ExactBoost, that boosts stumps to the actual loss function. By developing a novel extension of margin theory to the non-decomposable setting, it is possible to provably bound the generalization error of ExactBoost for many important metrics with different levels of non-decomposability. Through extensive examples, it is shown that such theoretical guarantees translate to competitive empirical performance. In particular, when used as an ensembler, ExactBoost is able to significantly outperform other surrogate-based and exact algorithms available.

Chat is not available.