Self-Attention Message Passing for Contrastive Few-Shot Learning

Ojas Kishorkumar Shirekar, Anuj Singh, Hadi Jamali-Rad

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

3 Downloads (Pure)


Humans have a unique ability to learn new representations from just a handful of examples with little to no supervision. Deep learning models, however, require an abundance of data and supervision to perform at a satisfactory level. Unsupervised few-shot learning (U-FSL) is the pursuit of bridging this gap between machines and humans. Inspired by the capacity of graph neural networks (GNNs) in discovering complex inter-sample relationships, we propose a novel self-attention based message passing contrastive learning approach (coined as SAMP-CLR) for U-FSL pre-training. We also propose an optimal transport (OT) based fine-tuning strategy (we call OpT-Tune) to efficiently induce task awareness into our novel end-to-end unsupervised few-shot classification framework (SAMPTransfer). Our extensive experimental results corroborate the efficacy of SAMPTransferin a variety of downstream few-shot classification scenarios, setting a new state-of-the-art for U-FSL on both miniImageNet and tieredImageNet benchmarks, offering up to 7%+ and 5%+ improvements, respectively. Our further investigations also confirm that SAMPTransferremains on-par with some supervised baselines on miniImageNet and outperforms all existing U-FSL baselines in a challenging cross-domain scenario. Our code can be found in our GitHub repository:
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
EditorsLisa O’Conner
Place of PublicationPiscataway
Number of pages11
ISBN (Electronic)978-1-6654-9346-8
ISBN (Print)978-1-6654-9347-5
Publication statusPublished - 2023
EventWACV: 2023 IEEE Winter Conference on Applications of Computer Vision - Waikoloa, United States
Duration: 2 Jan 20237 Jan 2023


Country/TerritoryUnited States
City Waikoloa

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Cite this