Skip to yearly menu bar Skip to main content


Learning to Defer to a Population: A Meta-Learning Approach

Dharmesh Tailor · Aditya Patra · Rajeev Verma · Putra Manggala · Eric Nalisnick

MR1 & MR2 - Number 33
award Student Paper Highlight
[ ]
Fri 3 May 8 a.m. PDT — 8:30 a.m. PDT
Oral presentation: Oral: General Machine Learning
Fri 3 May 7 a.m. PDT — 8 a.m. PDT


The learning to defer (L2D) framework allows autonomous systems to be safe and robust by allocating difficult decisions to a human expert. All existing work on L2D assumes that each expert is well-identified, and if any expert were to change, the system should be re-trained. In this work, we alleviate this constraint, formulating an L2D system that can cope with never-before-seen experts at test-time. We accomplish this by using a meta-learning architecture for the deferral function: given a small context set to identify the currently available expert, the model can quickly adapt its deferral policy. We also employ an attention mechanism that is able to look for points in the context set that are similar to a given test point, leading to an even more precise assessment of the expert's abilities. In the experiments, we demonstrate the usefulness of this architecture on image recognition, traffic sign detection, and skin lesion diagnosis benchmarks.

Live content is unavailable. Log in and register to view live content