Group Activity Recognition via Spatio-Temporal Reasoning of Key Instances


Haoting He (Xi'an Jiaotong University), Yaochen Li (Xi'an Jiaotong University), Yutong Wang (Xi'an Jiaotong University), Gaojie Li (Xi'an Jiaotong University), Wei Guo (Xi'an Jiaotong University), Runlin Zou (Xi'an Jiaotong University)
The 35th British Machine Vision Conference

Abstract

The task of group activity recognition is to detect the group behavior performed by a group of people, and detecting the key actors and key frames is particularly important for judging group activity. Therefore, we propose a key instances based spatio-temporal reasoning model. The proposed key instance identification module can identify key roles and key frames from video sequences, and dynamically aggregate the features of related actors through a graph relationship reasoning model. Joint features and RGB features are extracted from the video sequence, and the two are fused through the proposed multi-modal fusion TCT module, which enhances the expressive ability of the original features. In order to infer group activity through spatio-temporal correlation, the improved cross-transformer module is further used to perform spatio-temporal synchronic reasoning on group activity from two dimensions: time and space. Experimental results demonstrate that our proposed method achieves high accuracy on two public general data sets, and outperforms most of state-of-the-art methods.

Citation

@inproceedings{He_2024_BMVC,
author    = {Haoting He and Yaochen Li and Yutong Wang and Gaojie Li and Wei Guo and Runlin Zou},
title     = {Group Activity Recognition via Spatio-Temporal Reasoning of Key Instances},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0047.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection