One-step time-dependent future video frame prediction with a convolutional encoder-decoder neural network

Vedran Vukotic, Silvia Pintea, Christian Raymond, Guillaume Gravier, Jan van Gemert

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

14 Citations (Scopus)

Abstract

There is an inherent need for autonomous cars, drones, and other robots to have a notion of how their environment behaves and to anticipate changes in the near future. In this work, we focus on anticipating future appearance given the current frame of a video. Existing work focuses on either predicting the future appearance as the next frame of a video, or predicting future motion as optical flow or motion trajectories starting from a single video frame. This work stretches the ability of CNNs (Convolutional Neural Networks) to predict an anticipation of appearance at an arbitrarily given future time, not necessarily the next video frame. We condition our predicted future appearance on a continuous time variable that allows us to anticipate future frames at a given temporal distance, directly from the input video frame. We show that CNNs can learn an intrinsic representation of typical appearance changes over time and successfully generate realistic predictions at a deliberate time difference in the near future.

Original languageEnglish
Title of host publicationImage Analysis and Processing - ICIAP 2017
Subtitle of host publication19th International Conference, Proceedings
EditorsS. Battiato, G. Gallo, R. Schettini, F. Stanco
Place of PublicationCham
PublisherSpringer
Pages140-151
Number of pages12
EditionPart 1
ISBN (Electronic)978-3-319-68560-1
ISBN (Print)978-3-319-68559-5
DOIs
Publication statusPublished - 2017
EventImage Analysis and Processing - ICIAP 2017 : 19th International Conference - Catania, Italy
Duration: 11 Sep 201715 Sep 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer International Publishing AG
Volume10484
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceImage Analysis and Processing - ICIAP 2017
CountryItaly
CityCatania
Period11/09/1715/09/17

Keywords

  • Action forecasting
  • Appearance prediction
  • CNNs
  • Future video frame prediction
  • Generative models
  • Scene understanding

Fingerprint Dive into the research topics of 'One-step time-dependent future video frame prediction with a convolutional encoder-decoder neural network'. Together they form a unique fingerprint.

  • Cite this

    Vukotic, V., Pintea, S., Raymond, C., Gravier, G., & van Gemert, J. (2017). One-step time-dependent future video frame prediction with a convolutional encoder-decoder neural network. In S. Battiato, G. Gallo, R. Schettini, & F. Stanco (Eds.), Image Analysis and Processing - ICIAP 2017 : 19th International Conference, Proceedings (Part 1 ed., pp. 140-151). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10484 ). Springer. https://doi.org/10.1007/978-3-319-68560-1_13