Objects do not disappear: Video object detection by single-frame object location anticipation

Xin Liu; Jan C. van Gemert; Fatemeh Karimi  Nejadasl; Olaf Booij; Silvia L. Pintea

doi:10.1109/ICCV51070.2023.00640

Objects do not disappear: Video object detection by single-frame object location anticipation

Xin Liu, Jan C. van Gemert, Fatemeh Karimi Nejadasl, Olaf Booij, Silvia L. Pintea

Pattern Recognition and Bioinformatics

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

12 Downloads (Pure)

Abstract

Objects in videos are typically characterized by continuous smooth motion. We exploit continuous smooth motion in three ways. 1) Improved accuracy by using object motion as an additional source of supervision, which we obtain by anticipating object locations from a static keyframe. 2) Improved efficiency by only doing the expensive feature computations on a small subset of all frames. Because neighboring video frames are often redundant, we only compute features for a single static keyframe and predict object locations in subsequent frames. 3) Reduced annotation cost, where we only annotate the keyframe and use smooth pseudo-motion between keyframes. We demonstrate computational efficiency, annotation efficiency, and improved mean average precision compared to the state-of-the-art on four datasets: ImageNet VID, EPIC KITCHENS-55, YouTube-BoundingBoxes and Waymo Open dataset. Our source code is available at https://github.com/L-KID/Video-object-detection-by-location-anticipation.

Original language	English
Title of host publication	Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Editors	Cristina Ceballos
Place of Publication	Piscataway
Publisher	IEEE
Pages	6927-6938
Number of pages	12
ISBN (Electronic)	979-8-3503-0718-4
ISBN (Print)	979-8-3503-0719-1
DOIs	https://doi.org/10.1109/ICCV51070.2023.00640
Publication status	Published - 2023
Event	2023 IEEE/CVF International Conference on Computer Vision (ICCV) - Paris, France Duration: 1 Oct 2023 → 6 Oct 2023

Publication series

Name	Proceedings of the IEEE International Conference on Computer Vision
ISSN (Print)	1550-5499

Conference

Conference	2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Country/Territory	France
City	Paris
Period	1/10/23 → 6/10/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Access to Document

10.1109/ICCV51070.2023.00640

Objects_do_not_disappear_Video_object_detection_by_single-frame_object_location_anticipationFinal published version, 4.99 MB

Cite this

Liu, X., van Gemert, J. C., Nejadasl, F. K., Booij, O., & Pintea, S. L. (2023). Objects do not disappear: Video object detection by single-frame object location anticipation. In C. Ceballos (Ed.), Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 6927-6938). (Proceedings of the IEEE International Conference on Computer Vision). IEEE. https://doi.org/10.1109/ICCV51070.2023.00640

Liu, Xin ; van Gemert, Jan C. ; Nejadasl, Fatemeh Karimi et al. / Objects do not disappear : Video object detection by single-frame object location anticipation. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). editor / Cristina Ceballos. Piscataway : IEEE, 2023. pp. 6927-6938 (Proceedings of the IEEE International Conference on Computer Vision).

@inproceedings{8b66a4c1eabe49a28af124ddd275ebbb,

title = "Objects do not disappear: Video object detection by single-frame object location anticipation",

abstract = "Objects in videos are typically characterized by continuous smooth motion. We exploit continuous smooth motion in three ways. 1) Improved accuracy by using object motion as an additional source of supervision, which we obtain by anticipating object locations from a static keyframe. 2) Improved efficiency by only doing the expensive feature computations on a small subset of all frames. Because neighboring video frames are often redundant, we only compute features for a single static keyframe and predict object locations in subsequent frames. 3) Reduced annotation cost, where we only annotate the keyframe and use smooth pseudo-motion between keyframes. We demonstrate computational efficiency, annotation efficiency, and improved mean average precision compared to the state-of-the-art on four datasets: ImageNet VID, EPIC KITCHENS-55, YouTube-BoundingBoxes and Waymo Open dataset. Our source code is available at https://github.com/L-KID/Video-object-detection-by-location-anticipation.",

author = "Xin Liu and {van Gemert}, {Jan C.} and Nejadasl, {Fatemeh Karimi} and Olaf Booij and Pintea, {Silvia L.}",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 2023 IEEE/CVF International Conference on Computer Vision (ICCV) ; Conference date: 01-10-2023 Through 06-10-2023",

year = "2023",

doi = "10.1109/ICCV51070.2023.00640",

language = "English",

isbn = "979-8-3503-0719-1",

series = "Proceedings of the IEEE International Conference on Computer Vision",

publisher = "IEEE",

pages = "6927--6938",

editor = "Ceballos, {Cristina }",

booktitle = "Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV)",

address = "United States",

}

Liu, X , van Gemert, JC, Nejadasl, FK, Booij, O & Pintea, SL 2023, Objects do not disappear: Video object detection by single-frame object location anticipation. in C Ceballos (ed.), Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). Proceedings of the IEEE International Conference on Computer Vision, IEEE, Piscataway, pp. 6927-6938, 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 1/10/23. https://doi.org/10.1109/ICCV51070.2023.00640

Objects do not disappear: Video object detection by single-frame object location anticipation. / Liu, Xin ; van Gemert, Jan C.; Nejadasl, Fatemeh Karimi et al.
Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). ed. / Cristina Ceballos. Piscataway: IEEE, 2023. p. 6927-6938 (Proceedings of the IEEE International Conference on Computer Vision).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Objects do not disappear

T2 - 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

AU - Liu, Xin

AU - van Gemert, Jan C.

AU - Nejadasl, Fatemeh Karimi

AU - Booij, Olaf

AU - Pintea, Silvia L.

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - Objects in videos are typically characterized by continuous smooth motion. We exploit continuous smooth motion in three ways. 1) Improved accuracy by using object motion as an additional source of supervision, which we obtain by anticipating object locations from a static keyframe. 2) Improved efficiency by only doing the expensive feature computations on a small subset of all frames. Because neighboring video frames are often redundant, we only compute features for a single static keyframe and predict object locations in subsequent frames. 3) Reduced annotation cost, where we only annotate the keyframe and use smooth pseudo-motion between keyframes. We demonstrate computational efficiency, annotation efficiency, and improved mean average precision compared to the state-of-the-art on four datasets: ImageNet VID, EPIC KITCHENS-55, YouTube-BoundingBoxes and Waymo Open dataset. Our source code is available at https://github.com/L-KID/Video-object-detection-by-location-anticipation.

AB - Objects in videos are typically characterized by continuous smooth motion. We exploit continuous smooth motion in three ways. 1) Improved accuracy by using object motion as an additional source of supervision, which we obtain by anticipating object locations from a static keyframe. 2) Improved efficiency by only doing the expensive feature computations on a small subset of all frames. Because neighboring video frames are often redundant, we only compute features for a single static keyframe and predict object locations in subsequent frames. 3) Reduced annotation cost, where we only annotate the keyframe and use smooth pseudo-motion between keyframes. We demonstrate computational efficiency, annotation efficiency, and improved mean average precision compared to the state-of-the-art on four datasets: ImageNet VID, EPIC KITCHENS-55, YouTube-BoundingBoxes and Waymo Open dataset. Our source code is available at https://github.com/L-KID/Video-object-detection-by-location-anticipation.

UR - http://www.scopus.com/inward/record.url?scp=85188238690&partnerID=8YFLogxK

U2 - 10.1109/ICCV51070.2023.00640

DO - 10.1109/ICCV51070.2023.00640

M3 - Conference contribution

SN - 979-8-3503-0719-1

T3 - Proceedings of the IEEE International Conference on Computer Vision

SP - 6927

EP - 6938

BT - Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

A2 - Ceballos, Cristina

PB - IEEE

CY - Piscataway

Y2 - 1 October 2023 through 6 October 2023

ER -

Liu X , van Gemert JC, Nejadasl FK, Booij O , Pintea SL. Objects do not disappear: Video object detection by single-frame object location anticipation. In Ceballos C, editor, Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE. 2023. p. 6927-6938. (Proceedings of the IEEE International Conference on Computer Vision). doi: 10.1109/ICCV51070.2023.00640

Objects do not disappear: Video object detection by single-frame object location anticipation

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this