The Unreasonable Effectiveness of Large Language-Vision Models for Source-free Video Domain Adaptation - Télécom Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

The Unreasonable Effectiveness of Large Language-Vision Models for Source-free Video Domain Adaptation

Giacomo Zara
  • Fonction : Auteur
Alessandro Conti
  • Fonction : Auteur
Subhankar Roy
  • Fonction : Auteur
  • PersonId : 1360564
Paolo Rota
  • Fonction : Auteur
Elisa Ricci
  • Fonction : Auteur

Résumé

Source-Free Video Unsupervised Domain Adaptation (SFVUDA) task consists in adapting an action recognition model, trained on a labelled source dataset, to an unlabelled target dataset, without accessing the actual source data. The previous approaches have attempted to address SFVUDA by leveraging self-supervision (e.g., enforcing temporal consistency) derived from the target data itself. In this work, we take an orthogonal approach by exploiting "web-supervision" from Large Language-Vision Models (LLVMs), driven by the rationale that LLVMs contain a rich world prior surprisingly robust to domain-shift. We showcase the unreasonable effectiveness of integrating LLVMs for SFVUDA by devising an intuitive and parameter-efficient method, which we name Domain Adaptation with Large Language-Vision models (DALL-V), that distills the world prior and complementary source model information into a student network tailored for the target. Despite the simplicity, DALL-V achieves significant improvement over state-of-the-art SFVUDA methods.

Dates et versions

hal-04205043 , version 1 (12-09-2023)

Identifiants

Citer

Giacomo Zara, Alessandro Conti, Subhankar Roy, Stéphane Lathuilière, Paolo Rota, et al.. The Unreasonable Effectiveness of Large Language-Vision Models for Source-free Video Domain Adaptation. International Conference on Computer Vision, Oct 2023, Paris, France. ⟨hal-04205043⟩
11 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More