Lightweight Human Pose Estimation with Enhanced Knowledge Review


Hao Xu (Nanjing University of Information Science and Technology), Shengye Yan (Nanjing University of Information Science and Technology), Wei Zheng (MINIEYE)
The 35th British Machine Vision Conference
PDFPoster

Abstract

While current state-of-the-art human pose estimation methods have demonstrated remarkable performance, they frequently suffer from a significant parameter and computation overhead, resulting in slow inference speeds. For this issue, we propose a novel approach to knowledge distillation in lightweight human pose estimation. In previous knowledge distillation methods, the strategies in the cross-stage distillation distillation overlooked semantic mismatches caused by the differing complexities of teacher and student networks, potentially leading to negative regularization. To address this issue, we propose a novel method based on the cross-stage knowledge distillation framework. In the cross-stage knowledge distillation process, we transform student features in different stages through multiple receptive field feature transformations, by expanding the receptive fields of student features to better align them to the receptive fields of teacher features. We compute the similarity matrix between student and teacher features. By associating the features of both, we obtain cross-attention weights to facilitate effective cross-layer distillation interaction. At the output stage of the model, we replace the heatmap-based keypoint representation method with a classification coordinate-based approach, reducing the inference memory by 20\% and speeding up inference time. Additionally, the vanilla knowledge distillation is performed on the output horizontal and vertical coordinates. Extensive experiments on the MPII and COCO datasets validate the effectiveness of our approach.

Citation

@inproceedings{Xu_2024_BMVC,
author    = {Hao Xu and Shengye Yan and Wei Zheng},
title     = {Lightweight Human Pose Estimation with Enhanced Knowledge Review},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0427.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection