Budget-aware Dynamic Spatially Adaptive Inference


Georgios Zampokas (Imperial College London), Christos-Savvas Bouganis (Imperial College London), Dimitris Tzovaras (Centre for Research and Technology Hellas)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Availability of computational and memory resources can fluctuate in real systems, underscoring the utility of vision systems that can adapt to varying computational budgets. Spatially adaptive dynamic methods have emerged as a prominent solution, allowing the adaptation of deep learning models to meet various computation constraints. However, to perform accurate inference for multiple computation budgets, they either incur significant parameter overheads or scale poorly away from budgets they have been trained for. In light of this, our work addresses the problem of inference at multiple target computational budgets using spatially adaptive processing, under both performance and number of parameters constraints. Given a base model, we propose a spatially adaptive optimization framework, which equips the model with the ability to operate under $N$ target FLOP budgets. Different from relevant optimization approaches which typically train a full model for each specific budget, we leverage a single backbone while adapting only a few parameters for each target budget. This leads to negligible parameter overheads ($<$1\%) for each additional computation target. Our method is able to operate under multiple performance to FLOPs trade-offs and achieve similar performance to relevant state-of-the-art compression methods requiring significantly fewer extra parameters.

Citation

@inproceedings{Zampokas_2024_BMVC,
author    = {Georgios Zampokas and Christos-Savvas Bouganis and Dimitris Tzovaras},
title     = {Budget-aware Dynamic Spatially Adaptive Inference},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0853.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection