AISE: Adaptive Input Sampling for Explanation of Black-box Models


Evgeny Tsykunov (Intel Corporation), Wonju Lee (Intel Corporation), Minje Park (Intel)
The 35th British Machine Vision Conference
PDFPosterSupplementary

Abstract

White-box eXplainable AI (XAI) methods require access to gradients or model internal state, which might be impossible in the deployment framework settings, where the model is in optimized or compiled representation. On the other hand, black-box XAI methods might require thousands of model inferences. To address this, we introduce AISE, a novel method for derivative-free and efficient black-box model explanation. AISE requires an order of magnitude less number of inferences compared to existing gradient-free methods. To achieve this, we formulate saliency map generation as a kernel density estimation (KDE) problem, and adaptively sample input masks using a derivative-free optimizer to maximize mask saliency score. This adaptive sampling mechanism significantly improves the efficiency of input mask generation and thus increases convergence speed. AISE is designed to be task-agnostic and can be applied to a wide range of classification and object detection architectures. We show that AISE achieves state-of-the-art results, having just 300 model inferences or less (VOC dataset), while RISE requires 8000 and 5000 inferences for classification and detection respectively. Hence, AISE can be the best option for the efficient post-deployment explanation method for black-box models.

Citation

@inproceedings{Tsykunov_2024_BMVC,
author    = {Evgeny Tsykunov and Wonju Lee and Minje Park},
title     = {AISE: Adaptive Input Sampling for Explanation of Black-box Models},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0150.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection