Skip to yearly menu bar Skip to main content


Poster

Deep Classifier Mimicry without Data Access

Steven Braun · Martin Mundt · Kristian Kersting

MR1 & MR2 - Number 117
award Student Paper Highlight
[ ] [ Project Page ]
[ Slides [ Poster
Sat 4 May 6 a.m. PDT — 8:30 a.m. PDT
 
Oral presentation: Oral: Deep Learning
Sat 4 May 1:30 a.m. PDT — 2:30 a.m. PDT

Abstract:

Access to pre-trained models has recently emerged as a standard across numerous machine learning domains. Unfortunately, access to the original data the models were trained on may not equally be granted. This makes it tremendously challenging to fine-tune, compress models, adapt continually, or to do any other type of data-driven update. We posit that original data access may however not be required. Specifically, we propose Contrastive Abductive Knowledge Extraction (CAKE), a model-agnostic knowledge distillation procedure that mimics deep classifiers without access to the original data. To this end, CAKE generates pairs of noisy synthetic samples and diffuses them contrastively toward a model’s decision boundary. We empirically corroborate CAKE's effectiveness using several benchmark datasets and various architectural choices, paving the way for broad application.

Chat is not available.