Spatiotemporal Vision Transformer for Weakly Supervised Dense Prediction of Dynamic Brain Maps


Behnam Kazemivash (Georgia State University), Armin Iraji (Georgia State University), Sergey Plis (Georgia State University), Vince Calhoun (Georgia State University)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Dynamic brain maps are crucial for comprehending brain dynamism, involving the study of rapid changes in brain activity across different regions over time. However, computational neuroscience currently lacks effective methods for mapping these dynamics across multiple 4D temporally evolving brain networks. To address this issue, we propose a novel weakly supervised spatiotemporal dense prediction model for generating dynamic brain maps. We leveraged a Vision Transformer (ViT) as the backbone of the model to encode both spatial and temporal information for generation of individualized and independent 4D dynamic representations from input fMRI data, which vividly delineates brain networks. We utilized spatially constrained windowed ICA components extracted from the input fMRI as weak supervision for training the model, given the absence of ground-truth data. Our experiments, conducted on several large resting fMRI datasets, revealed temporal and inter-subject variations in the generated 4D maps. We also observed statistically significant differences in temporally averaged maps within the parietal cortex, as well as differences in the summation of the temporal gradient within the cerebellum, which also distinguish between individuals diagnosed with schizophrenia and controls.

Citation

@inproceedings{Kazemivash_2024_BMVC,
author    = {Behnam Kazemivash and Armin Iraji and Sergey Plis and Vince Calhoun},
title     = {Spatiotemporal Vision Transformer for Weakly Supervised Dense Prediction of Dynamic Brain Maps},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0932.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection