Hierarchical Sensor Fusion for Micro-Gesture Recognition with Pressure Sensor Array and Radar

H. Li, X. Liang, A. Shrestha, Y. Liu, H. Heidari, J. Le Kernec, F. Fioranelli

Research output: Contribution to journalArticleScientificpeer-review

12 Citations (Scopus)

Abstract

This paper presents a hierarchical sensor fusion approach for human micro-gesture recognition by combining an Ultra Wide Band (UWB) Doppler radar and wearable pressure sensors. First, the wrist-worn pressure sensor array (PSA) and Doppler radar are used to respectively identify static and dynamic gestures through a Quadratic-kernel SVM (Support Vector Machine) classifier. Then, a robust wrapper method is applied on the features from both sensors to search the optimal combination. Subsequently, two hierarchical approaches where one sensor acts as 'enhancer' of the other are explored. In the first case, scores from Doppler radar related to the confidence level of its classifier and the prediction label corresponding to the posterior probabilities are utilized to maximize the static hand gestures classification performance by hierarchical combination with PSA data. In the second case, the PSA acts as an 'enhancer' for radar to improve the dynamic gesture recognition. In this regard, different weights of the 'enhancer' sensor in the fusion process have been evaluated and compared in terms of classification accuracy. A realistic cross-validation method is chosen to test one unknown participant with the model trained by data from others, demonstrating that this hierarchical fusion approach for static and dynamic gestures yields approximately 15% improvement in classification accuracy in the best cases.

Original languageEnglish
Article number8882286
Pages (from-to)225-232
Number of pages8
JournalIEEE Journal of Electromagnetics, RF and Microwaves in Medicine and Biology
Volume4
Issue number3
DOIs
Publication statusPublished - 2020
Externally publishedYes

Keywords

  • Multimodal sensing
  • UWB Doppler radar
  • gesture classification
  • machine learning

Fingerprint

Dive into the research topics of 'Hierarchical Sensor Fusion for Micro-Gesture Recognition with Pressure Sensor Array and Radar'. Together they form a unique fingerprint.

Cite this