Spatial-Temporal NAS for Fast Surgical Segmentation


Matthew Lee (Medtronic), Felix John Samuel Bragman (Medtronic), Ricardo Sanchez-Matilla (Medtronic), Imanol Luengo (Medtronic), Danail Stoyanov (University College London)
The 35th British Machine Vision Conference
PDFPosterSupplementary

Abstract

Real time surgical video semantic segmentation requires accurate per-frame perfor- mance, inter-frame temporal consistency and high inference speeds. Additionally surgi- cal guidance systems require a combination of spatial and temporal information to pre- vent distracting temporal effects such as flickering. Designing models that learn effective spatial-temporal representations poses two challenges. First, the relative importance of temporal and spatial features given an architecture and a dataset is unknown, requiring human intuition to design the model. Secondly, adding temporal information greatly increases model size, which can negatively affect its inference speed and may lead to overfitting. We propose ST-NAS, a novel Neural Architecture Search (NAS) framework for optimising the balance between spatial and temporal operations in spatial-temporal models. We introduce a regulariser that promotes faster inference speeds whilst balanc- ing model performance. Components in the framework can be selectively used when considering the speed-accuracy requirements of the final model. We apply this frame- work to a private Partial Nephrectomy dataset and the public CholecSeg8K dataset. The model discovered through ST-NAS achieved a significant inference speedup (50-154%) with a marginal reduction in segmentation performance (1-5%). Experiments showed that the architecture discovered through ST-NAS required minimal temporal operations; supporting the effectiveness of architecture search in spatial-temporal network design.

Citation

@inproceedings{Lee_2024_BMVC,
author    = {Matthew Lee and Felix John Samuel Bragman and Ricardo Sanchez-Matilla and Imanol Luengo and Danail Stoyanov},
title     = {Spatial-Temporal NAS for Fast Surgical Segmentation},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0042.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection