DRAFT: Direct Radiance Fields Editing with Composable Operations


Zhihan Cai (Tsinghua University, Tsinghua University), Kailu Wu (Tsinghua University, Tsinghua University), Dapeng Cao (Xi'an Jiaotong University), Feng Chen (University of Hong Kong), Kaisheng Ma (Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Neural implicit representations, especially neural radiance fields (NeRF), have shown promising results in novel view synthesis of 3D scenes. However, radiance fields often implicitly model the 3D geometry, making direct manipulation challenging for users. Current editing methods often rely on customized NeRFs or employ ray bending, which lacks universality. To address these issues, we propose DRAFT, a novel editing system that translates user edits directly into implicit representations, bypassing the need for ray bending or customized labeling. DRAFT focuses on voxel-based radiance fields, mapping user edits to the latent space defined on the voxel representation. By modeling user edits as two steps - \textit{Selection} and \textit{Modification} - we implement various composable editing operations, including standard operations such as copy-and-paste, rotation, stretching, and removal. Additionally, for more realistic scene editing purposes, we introduce a stamp operation to address the artifacts caused by editing and seam carving on radiance fields to achieve content-aware resizing. Extensive experiments demonstrate that DRAFT produces realistic editing results for both synthetic and real-world scenes.

Citation

@inproceedings{Cai_2024_BMVC,
author    = {Zhihan Cai and Kailu Wu and Dapeng Cao and Feng Chen and Kaisheng Ma},
title     = {DRAFT: Direct Radiance Fields Editing with Composable Operations},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0037.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection