Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

17 Citations (Scopus)


Automatic kinship verification from facial information is a relatively new and open research problem in computer vision. This paper explores the possibility of learning an efficient facial representation for video-based kinship verification by exploiting the visual transformation between facial appearance of kin pairs. To this end, a Siamese-like coupled convolutional encoder-decoder network is proposed. To reveal resemblance patterns of kinship while discarding the similarity patterns that can also be observed between people who do not have a kin relationship, a novel contrastive loss function is defined in the visual appearance space. For further optimization, the learned representation is fine-tuned using a feature-based contrastive loss. An expression matching procedure is employed in the model to minimize the negative influence of expression differences between kin pairs. Each kin video is analyzed by a sliding temporal window to leverage short-term facial dynamics. The effectiveness of the proposed method is assessed on seven different kin relationships using smile videos of kin pairs. On the average, 93:65% verification accuracy is achieved, improving the state of the art.
Original languageEnglish
Title of host publication2017 IEEE International Conference on Computer Vision (ICCV)
EditorsL. O'Conner
Place of PublicationPiscataway
Number of pages10
ISBN (Electronic)978-1-5386-1032-9
ISBN (Print)978-1-5386-1033-6
Publication statusPublished - 2017
Event2017 IEEE International Conference on Computer Vision (ICCV) - Venice, Italy
Duration: 22 Oct 201729 Oct 2017


Conference2017 IEEE International Conference on Computer Vision (ICCV)
Abbreviated titleICCV 2017


  • Face
  • Visualization
  • Eyebrows
  • Feature extraction
  • Mouth
  • Computer vision
  • Optimization


Dive into the research topics of 'Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification'. Together they form a unique fingerprint.

Cite this