Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients


Maximilian Krahn (Aalto University), Michele Sasdelli (The University of Adelaide), Frances Fengyi Yang (University of Adelaide), Vladislav Golyanik (Saarland Informatics Campus, Max-Planck Institute for Informatics), Juho Kannala (Aalto University), Tat-Jun Chin (The University of Adelaide), Tolga Birdal (Imperial College London)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

We present Quantum Projected Stochastic Binary-Gradient Descent (QP-SBGD), a novel per-layer stochastic optimiser tailored towards training neural networks with binary weights, known as binary neural networks (BNNs), on quantum hardware. BNNs reduce the computational requirements and energy consumption of deep learning models with minimal loss in accuracy. However, training them in practice remains to be an open challenge. Most known BNN-optimisers either rely on projected updates or binarise weights post-training. Instead, QP-SBGD approximately maps the gradient onto binary variables, by solving a quadratic constrained binary optimisation. Moreover, we show how the NP-hard projection can be effectively executed on an adiabatic quantum annealer. We prove that if a fixed point exists in the binary variable space, the updates will converge to it. Our algorithm is implemented per layer, making it suitable for training larger networks on resource-limited quantum hardware. Through extensive evaluations, we show that QP-SBGD outperforms or is on par with competitive and well-established baselines such as BinaryConnect, signSGD and ProxQuant when optimising binary neural networks

Citation

@inproceedings{Krahn_2024_BMVC,
author    = {Maximilian Krahn and Michele Sasdelli and Frances Fengyi Yang and Vladislav Golyanik and Juho Kannala and Tat-Jun Chin and Tolga Birdal},
title     = {Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0111.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection