Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion


Zeyu Zhang (The Australian National University), Yiran Wang (University of Sydney, University of Sydney), Biao Wu (University of Technology Sydney), Shuo Chen (Monash University), Zhiyuan Zhang (University of Adelaide), SHIYA HUANG (University of Adelaide), Wenbo Zhang (University of Adelaide), Meng Fang (University of Liverpool), Ling Chen (University of Technology Sydney), Yang Zhao (La Trobe University)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

In recent years, there has been significant interest in creating 3D avatars and motions, driven by their diverse applications in areas like film-making, video games, AR/VR, and human-robot interaction. However, current efforts primarily concentrate on either generating the 3D avatar mesh alone or producing motion sequences, with integrating these two aspects proving to be a persistent challenge. Additionally, while avatar and motion generation predominantly target humans, extending these techniques to animals remains a significant challenge due to inadequate training data and methods. To bridge these gaps, our paper presents three key contributions. Firstly, we proposed a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars with motions through text queries. The method significantly advanced the progress in dynamic 3D character generation. Secondly, we introduced a LLM planner that coordinates both motion and avatar generation, which transforms a discriminative planning into a customizable Q&A fashion. Lastly, we presented an animal motion dataset named Zoo-300K, comprising approximately 300,000 text-motion pairs across 65 animal categories and its building pipeline ZooGen, which serves as a valuable resource for the community.

Citation

@inproceedings{Zhang_2024_BMVC,
author    = {Zeyu Zhang and Yiran Wang and Biao Wu and Shuo Chen and Zhiyuan Zhang and SHIYA HUANG and Wenbo Zhang and Meng Fang and Ling Chen and Yang Zhao},
title     = {Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0185.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection