GN-FR: Generalizable Neural Radinace Fields for Flare Removal


Gopi Raju Matta (Indian Institute of Technology Madras), Rahul Siddartha (Indian Institute of Technology Madras, Indian Institute of Technology, Madras), RONGALI SIMHACHALA VENKATA GIRISH (Indian Institute of Technology, Madras.), Sumit Sharma (Indian Institute of Technology, Madras), Kaushik Mitra (Indian Institute of Technology, Madras)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Flare, an optical phenomenon resulting from unwanted scattering and reflections within a lens system, presents a significant challenge in imaging. The diverse patterns of flares—such as halos, streaks, color bleeding, and haze—complicate the flare removal process. Existing traditional and learning-based methods have exhibited limited efficacy due to their reliance on single-image approaches, where flare removal is highly ill-posed. We address this by framing flare removal as a multi-view image problem, taking advan- tage of the view-dependent nature of flare artifacts. This approach leverages informa- tion from neighboring views to recover details obscured by flare in individual images. Our proposed framework, GN-FR (Generalizable Neural Radiance Fields for Flare Re- moval), can render flare-free views from a sparse set of input images affected by lens flare and generalizes across different scenes in an unsupervised manner. GN-FR incorporates several modules within the Generalizable NeRF Transformer (GNT) framework: Flare- occupancy Mask Generation (FMG), View Sampler (VS), and Point Sampler (PS). To overcome the impracticality of capturing both flare-corrupted and flare-free data, we in- troduce a masking loss function that utilizes mask information in an unsupervised setting. Additionally, we present the first-of-its-kind 3D multi-view flare dataset, comprising 17 real flare scenes with 782 images, 80 real flare patterns, and their corresponding anno- tated flare-occupancy masks. To our knowledge, this is the first work to address flare removal within a Neural Radiance Fields (NeRF) framework.

Citation

@inproceedings{Matta_2024_BMVC,
author    = {Gopi Raju Matta and Rahul Siddartha and RONGALI SIMHACHALA VENKATA GIRISH and Sumit Sharma and Kaushik Mitra},
title     = {GN-FR: Generalizable Neural Radinace Fields for Flare Removal},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0659.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection