Alignment-aware Patch-level Routing for Dynamic Video Frame Interpolation


Ban Chen (Samsung Electronics (China) R&D Center), Xin Jin (Samsung R&D Institute China-Nanjing (SRC-N)), LONG HAI WU (University of Science and Technology of China), Jie Chen (SRCN), Ilhyun Cho (Samsung), Cheul-hee Hahm (Samsung)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Existing dynamic Video Frame Interpolation (VFI) methods either struggle to produce satisfactory results on high-resolution videos due to the lack of explicit image warping, or fail to achieve practical speedup due to the use of inefficient sparse convolution. In this work, we propose a novel dynamic VFI architecture that combines image warping and patch-wise processing: the warping operation can compensate for the large motion in high-resolution videos, while patch-wise dynamic processing can be implemented with efficient regular convolution. Specifically, we develop an Alignment-aware Patch-wise Routing module to select the adaptive sub-networks to synthesize target patches from local patches in warped input frames. Extensive experiments demonstrate that our dynamic VFI method can achieve excellent results on 2K and 4K videos, saving considerable amount of computation and runtime with marginal performance degradation.

Citation

@inproceedings{Chen_2024_BMVC,
author    = {Ban Chen and Xin Jin and LONG HAI WU and Jie Chen and Ilhyun Cho and Cheul-hee Hahm},
title     = {Alignment-aware Patch-level Routing for Dynamic Video Frame Interpolation},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0023.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection