Insights from a Large-Scale Database of Material Depictions in Paintings

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

Deep learning has paved the way for strong recognition systems which are often both trained on and applied to natural images. In this paper, we examine the give-and-take relationship between such visual recognition systems and the rich information available in the fine arts. First, we find that visual recognition systems designed for natural images can work surprisingly well on paintings. In particular, we find that interactive segmentation tools can be used to cleanly annotate polygonal segments within paintings, a task which is time consuming to undertake by hand. We also find that FasterRCNN, a model which has been designed for object recognition in natural scenes, can be quickly repurposed for detection of materials in paintings. Second, we show that learning from paintings can be beneficial for neural networks that are intended to be used on natural images. We find that training on paintings instead of natural images can improve the quality of learned features and we further find that a large number of paintings can be a valuable source of test data for evaluating domain adaptation algorithms. Our experiments are based on a novel large-scale annotated database of material depictions in paintings which we detail in a separate manuscript.

Original languageEnglish
Title of host publicationPattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings
EditorsAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
PublisherSpringer
Pages531-545
Number of pages15
ISBN (Print)9783030687953
DOIs
Publication statusPublished - 2021
Event25th International Conference on Pattern Recognition Workshops, ICPR 2020 - Virtual, Online, Italy
Duration: 10 Jan 202115 Jan 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12663 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th International Conference on Pattern Recognition Workshops, ICPR 2020
CountryItaly
CityVirtual, Online
Period10/01/2115/01/21

Keywords

  • Artistic material depictions
  • Classification
  • Domain adaptation
  • Interpretability
  • Large-scale data
  • Segmentation

Fingerprint Dive into the research topics of 'Insights from a Large-Scale Database of Material Depictions in Paintings'. Together they form a unique fingerprint.

Cite this