Robust Anomaly Detection on Unreliable Data

Zilong Zhao, Sophie Cerf, Robert Birke, Bogdan Robu, Sara Bouchenak, Sonia Ben Mokhtar, Lydia Y. Chen

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

9 Citations (Scopus)

Abstract

Classification algorithms have been widely adopted to detect anomalies for various systems, e.g., IoT and cloud, under the common assumption that the data source is clean, i.e., features and labels are correctly set. However, data collected from the field can be unreliable due to careless annotations or malicious data transformation for incorrect anomaly detection. In this paper, we present a two-layer learning framework for robust anomaly detection (RAD) in the presence of unreliable anomaly labels. The first layer of quality model filters the suspicious data, where the second layer of classification model detects the anomaly types. We specifically focus on two use cases, (i) detecting 10 classes of IoT attacks and (ii) predicting 4 classes of task failures of big data jobs. Our evaluation results show that RAD can robustly improve the accuracy of anomaly detection, to reach up to 98% for IoT device attacks (i.e., +11%) and up to 83% for cloud task failures (i.e., +20%), under a significant percentage of altered anomaly labels. Index Terms-Unreliable Data; Anomaly Detection; Failures; Attacks; Machine Learning.

Original languageEnglish
Title of host publicationProceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019
Subtitle of host publicationProceedings
Place of PublicationPiscataway
PublisherIEEE
Pages630-637
Number of pages8
ISBN (Electronic)9781728100562
ISBN (Print)978-1-7281-0058-6
DOIs
Publication statusPublished - 2019
Event49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019 - Portland, United States
Duration: 24 Jun 201927 Jun 2019

Publication series

NameProceedings - 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019

Conference

Conference49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2019
CountryUnited States
CityPortland
Period24/06/1927/06/19

Keywords

  • Anomaly Detection
  • Attacks
  • Failures
  • Machine Learning
  • Unreliable Data

Fingerprint

Dive into the research topics of 'Robust Anomaly Detection on Unreliable Data'. Together they form a unique fingerprint.

Cite this