Human-in-the-Loop Rule Discovery for Micropost Event Detection

Akansha Bhardwaj, Jie Yang*, Philippe Cudré-Mauroux

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

11 Downloads (Pure)

Abstract

Platforms such as Twitter are increasingly being used for real-world event detection. Recent work often leverages event-related keywords for training machine learning based event detection models. These approaches make strong assumptions on the distribution of the relevant microposts containing the keyword – referred to as the expectation – and use it as a posterior regularization parameter during model training. Such approaches are, however, limited by the informativeness of the keywords and by the accuracy of the expectation estimation for keywords. In this work, we introduce a human-in-the-loop approach to jointly discover informative rules for model training while estimating their expectation. Our approach iteratively leverages the crowd to estimate both rule-specific expectation and the disagreement between the crowd and the model in order to discover new rules that are most beneficial for model training. To identify such rules, we introduce a hybrid human-machine workflow that engages human workers in rule discovery through an interactive hypothesis creation and testing interface and leverages automatic methods for suggesting useful rules for human verification. We empirically demonstrate the merits of our approach, on multiple real-world datasets and show that our approach improves the state of the art by a margin of 25.63% in terms of AUC.
Original languageEnglish
Pages (from-to)8100-8111
Number of pages12
JournalIEEE Transactions on Knowledge and Data Engineering
Volume35
Issue number8
DOIs
Publication statusPublished - 2022

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Event Detection
  • Human-in-the-loop AI
  • Rules in Machine Learning
  • Interactive Machine Learning

Fingerprint

Dive into the research topics of 'Human-in-the-Loop Rule Discovery for Micropost Event Detection'. Together they form a unique fingerprint.

Cite this