Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

23 Citations (Scopus)

Abstract

Automatic kinship verification from facial information is a relatively new and open research problem in computer vision. This paper explores the possibility of learning an efficient facial representation for video-based kinship verification by exploiting the visual transformation between facial appearance of kin pairs. To this end, a Siamese-like coupled convolutional encoder-decoder network is proposed. To reveal resemblance patterns of kinship while discarding the similarity patterns that can also be observed between people who do not have a kin relationship, a novel contrastive loss function is defined in the visual appearance space. For further optimization, the learned representation is fine-tuned using a feature-based contrastive loss. An expression matching procedure is employed in the model to minimize the negative influence of expression differences between kin pairs. Each kin video is analyzed by a sliding temporal window to leverage short-term facial dynamics. The effectiveness of the proposed method is assessed on seven different kin relationships using smile videos of kin pairs. On the average, 93:65% verification accuracy is achieved, improving the state of the art.
Original languageEnglish
Title of host publication2017 IEEE International Conference on Computer Vision (ICCV)
EditorsL. O'Conner
Place of PublicationPiscataway
PublisherIEEE
Pages2478-2487
Number of pages10
ISBN (Electronic)978-1-5386-1032-9
ISBN (Print)978-1-5386-1033-6
DOIs
Publication statusPublished - 2017
Event2017 IEEE International Conference on Computer Vision (ICCV) - Venice, Italy
Duration: 22 Oct 201729 Oct 2017

Conference

Conference2017 IEEE International Conference on Computer Vision (ICCV)
Abbreviated titleICCV 2017
Country/TerritoryItaly
CityVenice
Period22/10/1729/10/17

Keywords

  • Face
  • Visualization
  • Eyebrows
  • Feature extraction
  • Mouth
  • Computer vision
  • Optimization

Fingerprint

Dive into the research topics of 'Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification'. Together they form a unique fingerprint.

Cite this