MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds


Ziqiang Dang (Alibaba Group), Tianxing Fan (Zhejiang University), Boming Zhao (Zhejiang University), Xujie Shen (Zhejiang University), 王 磊 (Guangdong OPPO Mobile Telecommunications Corp.,Ltd.), Guofeng Zhang (Zhejiang University), Zhaopeng Cui (Zhejiang University)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Incorporating temporal information effectively is important for accurate 3D human motion estimation and generation which have wide applications from human-computer interaction to AR/VR. In this paper, we present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Different from existing mathematical or VAE-based methods, our representation is designed based on the neural distance field, which makes human dynamics explicitly quantified to a score and thus can measure human motion plausibility. Specifically, we propose novel decoupled joint acceleration manifolds to model human dynamics from existing limited motion data. Moreover, we introduce a novel optimization method using the manifold distance as guidance, which facilitates a variety of motion-related tasks. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks such as denoising real-world human mocap data, recovering human motion from partial 3D observations, mitigating jitters for SMPL-based pose estimators, and refining the results of motion in-betweening.

Citation

@inproceedings{Dang_2024_BMVC,
author    = {Ziqiang Dang and Tianxing Fan and Boming Zhao and Xujie Shen and 王 磊 and Guofeng Zhang and Zhaopeng Cui},
title     = {MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0108.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection