SagaGAN: Style Applied using Gram matrix Attribution based on StarGAN v2


Yongseon Yoo (Hanyang University), Seonggyu Kim (Hanyang University), Jong-Min Lee (Hanyang University)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Image-to-image translation aims to convert an image from one domain to another while preserving its content. AdaIN (Adaptive Instance Normalization) is a widely used style application method, but it may not fully capture the fine-grained visual characteristics of complex styles. We propose SagaGAN, a novel approach that combines the gram matrix with AdaIN to better capture and transfer style information. We introduce two loss functions: G1 loss and G2 loss, which focus on the differences between gram matrices of the style, generated, and input images. These losses enable SagaGAN to learn richer style information. Additionally, we incorporate a perceptual loss alongside the cycle consistency loss to maintain a balance between style application and content preservation. Experimental results demonstrate that SagaGAN effectively applies style information, leading to improved image generation performance compared to existing models. By leveraging the gram matrix to capture complex style characteristics while preserving content, SagaGAN enhances the style transfer capabilities of models like StarGAN v2.

Citation

@inproceedings{Yoo_2024_BMVC,
author    = {Yongseon Yoo and Seonggyu Kim and Jong-Min Lee},
title     = {SagaGAN: Style Applied using Gram matrix Attribution based on StarGAN v2},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0076.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection