An advanced active vision system with multimodal visual odometry perception for humanoid robots

Xin Wang, Pieter Jonker

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)


Using active vision to perceive surroundings instead of just passively receiving information, humans develop the ability to explore unknown environments. Humanoid robot active vision research has already half a century history. It covers comprehensive research areas and plenty of studies have been done. Nowadays, the new trend is to use a stereo setup or a Kinect with neck movements to realize active vision. However, human perception is a combination of eye and neck movements. This paper presents an advanced active vision system that works in a similar way as human vision. The main contributions are: a design of a set of controllers that mimic eye and neck movements, including saccade eye movements, pursuit eye movements, vestibulo-ocular reflex eye movements and vergence eye movements; an adaptive selection mechanism based on properties of objects to automatically choose an optimal tracking algorithm; a novel Multimodal Visual Odometry Perception method that combines stereopsis and convergence to enable robots to perform both precise action in action space and scene exploration in personal space. Experimental results prove the effectiveness and robustness of our system. Besides, the system works in real-time constraints with low-cost cameras and motors, providing an affordable solution for industrial applications.

Original languageEnglish
Article number1750006
Number of pages27
JournalInternational Journal of Humanoid Robotics
Issue number3
Publication statusPublished - 2017


  • Active vision
  • humanoid robots
  • saccade eye movements
  • pursuit eye movements
  • vestibulo-ocular re°ex (VOR) eye movements
  • vergence eye movements
  • adaptive tracking selection mechanism
  • multimodal visual odometry perception (MVOP)
  • stereopsis
  • convergence


Dive into the research topics of 'An advanced active vision system with multimodal visual odometry perception for humanoid robots'. Together they form a unique fingerprint.

Cite this