Spot On: Action Localization from Pointly-Supervised Proposals

Pascal Mettes, Jan van Gemert, CGM Snoek

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

22 Citations (Scopus)

Abstract

We strive for spatio-temporal localization of actions in videos. The state-of-the-art relies on action proposals at test time and selects the best one with a classifier trained on carefully annotated box annotations. Annotating action boxes in video is cumbersome, tedious, and error prone. Rather than annotating boxes, we propose to annotate actions in video with points on a sparse subset of frames only. We introduce an overlap measure between action proposals and points and incorporate them all into the objective of a non-convex Multiple Instance Learning optimization. Experimental evaluation on the UCF Sports and UCF 101 datasets shows that (i) spatio-temporal proposals can be used to train classifiers while retaining the localization performance, (ii) point annotations yield results comparable to box annotations while being significantly faster to annotate, (iii) with a minimum amount of supervision our approach is competitive to the state-of-the-art. Finally, we introduce spatio-temporal action annotations on the train and test videos of Hollywood2, resulting in Hollywood2Tubes, available at http://tinyurl.com/hollywood2tubes.
Original languageEnglish
Title of host publicationComputer Vision ECCV 2016
Subtitle of host publication14th European Conference, proceedings
EditorsB. Leibe, J. Matas, N. Sebe, M. Welling
Place of PublicationCham
PublisherSpringer
Pages437-453
Number of pages17
Volume5
ISBN (Electronic)978-3-319-46454-1
ISBN (Print)978-3-319-46453-4
DOIs
Publication statusPublished - 2016
EventECCV 2016: 29th European Conference on Computer Vision - Amsterdam, Netherlands
Duration: 8 Oct 201616 Oct 2016

Publication series

NameLecture Notes in Computer Science
PublisherSpringer International Publishing AG
Volume9909
ISSN (Print)0302-9743

Conference

ConferenceECCV 2016
CountryNetherlands
CityAmsterdam
Period8/10/1616/10/16

Keywords

  • Action localization
  • Action proposals

Fingerprint Dive into the research topics of 'Spot On: Action Localization from Pointly-Supervised Proposals'. Together they form a unique fingerprint.

  • Cite this

    Mettes, P., van Gemert, J., & Snoek, CGM. (2016). Spot On: Action Localization from Pointly-Supervised Proposals. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision ECCV 2016: 14th European Conference, proceedings (Vol. 5, pp. 437-453). (Lecture Notes in Computer Science; Vol. 9909). Springer. https://doi.org/10.1007/978-3-319-46454-1_27