COSMo: CLIP Talks on Open-Set Multi-Target Domain Adaptation


Munish Monga (Indian Institute of Technology, Bombay), Sachin Kumar Giroh (Indian Institute of Technology, Bombay), Ankit Jha (The LNM Institute of Information Technology), Mainak Singha (Indian Institute of Technology, Bombay), Biplab Banerjee (Indian Institute of Technology, Bombay, Dhirubhai Ambani Institute Of Information and Communication Technology), Jocelyn Chanussot (INRIA)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Multi-Target Domain Adaptation (MTDA) entails learning domain-invariant information from a single source domain and applying it to multiple unlabeled target domains. Yet, existing MTDA methods predominantly focus on addressing domain shifts within visual features, often overlooking semantic features and struggling to handle unknown classes, resulting in what is known as Open-Set (OS) MTDA. While large-scale vision-language foundation models like CLIP show promise, their potential for MTDA remains largely unexplored. This paper introduces COSMo, a novel method that learns domain-agnostic prompts through source domain-guided prompt learning to tackle the MTDA problem in the prompt space. By leveraging a domain-specific bias network and separate prompts for known and unknown classes, COSMo effectively adapts across domain and class shifts. To the best of our knowledge, COSMo is the first method to address Open-Set Multi-Target DA (OSMTDA), offering a more realistic representation of real-world scenarios and addressing the challenges of both open-set and multi-target DA. COSMo demonstrates an average improvement of 5.1% across three challenging datasets: Mini-DomainNet, Office-31, and Office-Home, compared to other related DA methods adapted to operate within the OSMTDA setting.

Citation

@inproceedings{Monga_2024_BMVC,
author    = {Munish Monga and Sachin Kumar Giroh and Ankit Jha and Mainak Singha and Biplab Banerjee and Jocelyn Chanussot},
title     = {COSMo: CLIP Talks on Open-Set Multi-Target Domain Adaptation},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0031.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection