Hybrid connection and host clustering for community detection in spatial-temporal network data

M.P. Roeling, A. Nadeem, S.E. Verwer

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

11 Downloads (Pure)

Abstract

Network data clustering and sequential data mining are large
fields of research, but how to combine them to analyze spatial-temporal
network data remains a technical challenge. This study investigates a
novel combination of two sequential similarity methods (Dynamic Time
Warping and N-grams with Cosine distances), with two state-of-the-art
unsupervised network clustering algorithms (Hierarchical Density-based
Clustering and Stochastic Block Models). A popular way to combine such
methods is to first cluster the sequential network data, resulting in connection types. The hosts in the network can then be clustered conditioned
on these types. In contrast, our approach clusters nodes and edges in one
go, i.e., without giving the output of a first clustering step as input for a
second step. We achieve this by implementing sequential distances as covariates for host clustering. While being fully unsupervised, our method
outperforms many existing approaches. To the best of our knowledge, the
only approaches with comparable performance require manual filtering
of connections and feature engineering steps. In contrast, our method is
applied to raw network traffic. We apply our pipeline to the problem of
detecting infected hosts (network nodes) from logs of unlabelled network
traffic (sequential data). On data from the Stratosphere IPS project (CTUMalware-Capture-Botnet-91), which includes malicious (Conficker botnet) as well as benign hosts, we show that our method perfectly detects
peripheral, benign, and malicious hosts in different clusters. We replicate our results in the well-known ISOT dataset (Storm, Waledac, Zeus
botnets) with comparable performance: conjointly, 99.97% of nodes were
categorized correctly
Original languageEnglish
Title of host publicationEuropean Conference on Machine Learning
Subtitle of host publicationMachine Learning for Cybersecurity
Number of pages28
Publication statusPublished - 14 Sep 2020
Event2nd Workshop on
machine learning for cybersecurity
- Ghent, Belgium
Duration: 14 Sep 202014 Sep 2020

Conference

Conference2nd Workshop on
machine learning for cybersecurity
CountryBelgium
CityGhent
Period14/09/2014/09/20

Fingerprint Dive into the research topics of 'Hybrid connection and host clustering for community detection in spatial-temporal network data'. Together they form a unique fingerprint.

  • Cite this

    Roeling, M. P., Nadeem, A., & Verwer, S. E. (2020). Hybrid connection and host clustering for community detection in spatial-temporal network data. In European Conference on Machine Learning: Machine Learning for Cybersecurity