RESTAD: Reconstruction and Similarity Based Transformer for Time Series Anomaly Detection

Ramin Ghorbani; Marcel J.T. Reinders; David M. J. Tax

doi:10.1109/MLSP58920.2024.10734755

RESTAD: Reconstruction and Similarity Based Transformer for Time Series Anomaly Detection

Ramin Ghorbani, Marcel J.T. Reinders, David M. J. Tax

Pattern Recognition and Bioinformatics

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

Abstract

Anomaly detection in time series data is crucial across various domains. The scarcity of labeled data for such tasks has increased the attention towards unsupervised learning methods. These approaches, often relying solely on reconstruction error, typically fail to detect subtle anomalies in complex datasets. To address this, we introduce RESTAD, an adaptation of the Transformer model by incorporating a layer of Radial Basis Function (RBF) neurons within its architecture. This layer fits a non-parametric density in the latent representation, such that a high RBF output indicates similarity with predominantly normal training data. RESTAD integrates the RBF similarity scores with the reconstruction errors to increase sensitivity to anomalies. Our empirical evaluations demonstrate that RESTAD outperforms various established baselines across multiple benchmark datasets.

Original language	English
Title of host publication	Proceedings of the 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)
Place of Publication	Danvers
Publisher	IEEE
Number of pages	6
ISBN (Electronic)	979-8-3503-7225-0
ISBN (Print)	979-8-3503-7226-7
DOIs	https://doi.org/10.1109/MLSP58920.2024.10734755
Publication status	Published - 2024
Event	2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP) - London, United Kingdom Duration: 22 Sept 2024 → 25 Sept 2024 Conference number: 34th

Conference

Conference	2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)
Country/Territory	United Kingdom
City	London
Period	22/09/24 → 25/09/24

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Time Series
Anomaly Detection
Radial Basis Function (RBF) kernel
Transformer

Access to Document

10.1109/MLSP58920.2024.10734755

Embargoed Document

RESTAD_Reconstruction_and_Similarity_Based_Transformer_for_Time_Series_Anomaly_Detection
Final published version, 765 KB
Embargo ends: 4/05/25

Cite this

@inproceedings{91ade70ddb6a413d961fdceb06ec370e,

title = "RESTAD: Reconstruction and Similarity Based Transformer for Time Series Anomaly Detection",

abstract = "Anomaly detection in time series data is crucial across various domains. The scarcity of labeled data for such tasks has increased the attention towards unsupervised learning methods. These approaches, often relying solely on reconstruction error, typically fail to detect subtle anomalies in complex datasets. To address this, we introduce RESTAD, an adaptation of the Transformer model by incorporating a layer of Radial Basis Function (RBF) neurons within its architecture. This layer fits a non-parametric density in the latent representation, such that a high RBF output indicates similarity with predominantly normal training data. RESTAD integrates the RBF similarity scores with the reconstruction errors to increase sensitivity to anomalies. Our empirical evaluations demonstrate that RESTAD outperforms various established baselines across multiple benchmark datasets.",

keywords = "Time Series, Anomaly Detection, Radial Basis Function (RBF) kernel, Transformer",

author = "Ramin Ghorbani and Reinders, {Marcel J.T.} and Tax, {David M. J.}",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP) ; Conference date: 22-09-2024 Through 25-09-2024",

year = "2024",

doi = "10.1109/MLSP58920.2024.10734755",

language = "English",

isbn = "979-8-3503-7226-7",

booktitle = "Proceedings of the 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)",

publisher = "IEEE",

address = "United States",

}

Ghorbani, R , Reinders, MJT & Tax, DMJ 2024, RESTAD: Reconstruction and Similarity Based Transformer for Time Series Anomaly Detection. in Proceedings of the 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, Danvers, 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP), London, United Kingdom, 22/09/24. https://doi.org/10.1109/MLSP58920.2024.10734755

RESTAD: Reconstruction and Similarity Based Transformer for Time Series Anomaly Detection. / Ghorbani, Ramin ; Reinders, Marcel J.T.; Tax, David M. J.
Proceedings of the 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP). Danvers: IEEE, 2024.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - RESTAD: Reconstruction and Similarity Based Transformer for Time Series Anomaly Detection

AU - Ghorbani, Ramin

AU - Reinders, Marcel J.T.

AU - Tax, David M. J.

N1 - Conference code: 34th

PY - 2024

Y1 - 2024

N2 - Anomaly detection in time series data is crucial across various domains. The scarcity of labeled data for such tasks has increased the attention towards unsupervised learning methods. These approaches, often relying solely on reconstruction error, typically fail to detect subtle anomalies in complex datasets. To address this, we introduce RESTAD, an adaptation of the Transformer model by incorporating a layer of Radial Basis Function (RBF) neurons within its architecture. This layer fits a non-parametric density in the latent representation, such that a high RBF output indicates similarity with predominantly normal training data. RESTAD integrates the RBF similarity scores with the reconstruction errors to increase sensitivity to anomalies. Our empirical evaluations demonstrate that RESTAD outperforms various established baselines across multiple benchmark datasets.

AB - Anomaly detection in time series data is crucial across various domains. The scarcity of labeled data for such tasks has increased the attention towards unsupervised learning methods. These approaches, often relying solely on reconstruction error, typically fail to detect subtle anomalies in complex datasets. To address this, we introduce RESTAD, an adaptation of the Transformer model by incorporating a layer of Radial Basis Function (RBF) neurons within its architecture. This layer fits a non-parametric density in the latent representation, such that a high RBF output indicates similarity with predominantly normal training data. RESTAD integrates the RBF similarity scores with the reconstruction errors to increase sensitivity to anomalies. Our empirical evaluations demonstrate that RESTAD outperforms various established baselines across multiple benchmark datasets.

KW - Time Series

KW - Anomaly Detection

KW - Radial Basis Function (RBF) kernel

KW - Transformer

U2 - 10.1109/MLSP58920.2024.10734755

DO - 10.1109/MLSP58920.2024.10734755

M3 - Conference contribution

SN - 979-8-3503-7226-7

BT - Proceedings of the 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)

PB - IEEE

CY - Danvers

T2 - 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)

Y2 - 22 September 2024 through 25 September 2024

ER -

RESTAD: Reconstruction and Similarity Based Transformer for Time Series Anomaly Detection

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Embargoed Document

Fingerprint

Cite this