Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation


Xiaoyue Mi (University of the Chinese Academy of Sciences), Fan Tang (Institute of Computing Technology, CAS), Yepeng Weng (Lenovo Group Limited), Danding Wang (Institute of Computing Technology, Chinese Academy of Sciences), Juan Cao (Institute of Computing Technology, Chinese Academy of Sciences), Sheng Tang (Institute of Computing Technology, Chinese Academy of Sciences), Peng Li (Tsinghua University), Yang Liu (Tsinghua University)
The 35th British Machine Vision Conference
PDFPosterSupplementary

Abstract

Despite the effectiveness in improving the robustness of neural networks, adversarial training has suffered from the natural accuracy degradation problem, i.e., accuracy on natural samples has reduced significantly. In this study, we reveal that natural accuracy degradation is highly related to the disruption of the natural sample topology in the representation space by quantitative and qualitative experiments. Based on this observation, we propose Topology-pReserving Adversarial traINing (TRAIN) to alleviate the problem by preserving the topology structure of natural samples from a standard model trained only on natural samples during adversarial training. As an additional regularization, our method can be combined with various popular adversarial training algorithms, taking advantage of both sides. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet show that our proposed method achieves consistent and significant improvements over various strong baselines in most cases. Specifically, without additional data, TRAIN achieves up to $\mathbf{8.86\%}$ improvement in natural accuracy and $\mathbf{6.33\%}$ improvement in robust accuracy.

Citation

@inproceedings{Mi_2024_BMVC,
author    = {Xiaoyue Mi and Fan Tang and Yepeng Weng and Danding Wang and Juan Cao and Sheng Tang and Peng Li and Yang Liu},
title     = {Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0168.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection