DEPICT: Diffusion-Enabled Permutation Importance for Image Classification Tasks

Sarah Jabbour1, Gregory Kondas1, Ella A. Kazerooni1, Michael W. Sjoding1, David Fouhey2*, Jenna Wiens1*
1University of Michigan, 2New York University
*Co-senior authors
Paper arXiv Code

Approach Overview In standard permutation importance for tabular data (left), one obtains feature importance by permuting each feature column and measuring the impact on model performance. In diffusion-enabled image permutation importance (right), text-conditioned diffusion models generate dataset images for classifier evaluation. Features are permuted in the model's conditioned text space. To validate results, one can further check that only the permuted concept changed.

Abstract

We propose a permutation-based explanation method for image classifiers. Current image model explanations like activation maps are limited to instance-based explanations in the pixel space, making it difficult to understand global model behavior. Permutation based explanations for tabular data classifiers measure feature importance by comparing original model performance to model performance on data after permuting a feature. We propose an explanation method for image-based models that permutes interpretable concepts across dataset images. Given a dataset of images labeled with specific concepts like captions, we permute a concept across examples and then generate images via a text-conditioned diffusion model. Model importance is then given by the change in classifier performance relative to unpermuted data. When applied to a set of concepts, the method generates a ranking of feature importance. We show that this approach recovers underlying model feature importance on synthetic and real-world image classification tasks.

Approach

In tabular-based permutation importance (left), one obtains feature importance by permuting each feature column and measuring the impact on model performance. In diffusion-enabled image permutation importance (right), text-conditioned diffusion models generate dataset images for classifier evaluation. Features are permuted in the model's conditioned text space. To validate results, one can further check that only the permuted concept changed.