Shadows, highlights and faces: The contribution of a 'human in the loop' to digital art history

While automatic computational techniques appear to reveal novel insights in digital art history, a complementary approach seems to get less attention: that of human annotation. We argue and exemplify that a 'human in the loop' can reveal insights that may be difficult to detect automatically. Specifically, we focused on perceptual aspects within pictorial art. Using rather simple annotation tasks (e.g. delineate human heights, indicate highlights and classify gaze direction) we could both replicate earlier findings and reveal novel insights into pictorial conventions. We found that Canaletto depicted human figures in rather accurate perspective, varied viewpoint elevation between approximately 3 and 9 m and highly preferred light directions parallel to the projection plane. Furthermore, we found that taking the averaged images of leftward-looking faces reveals a woman, and for rightward-looking faces showed a male, confirming earlier accounts on lateral gender bias in pictorial art. Lastly, we confirmed and refined the well-known light-from-the-top-left bias. Together, the annotations, analyses and results exemplify how manual annotations can contribute and complement to digital art history.