Ruling the Rules: Quantifying the Evolution of Rulesets, Alerts and Incidents in Network Intrusion Detection

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

86 Downloads (Pure)

Abstract

Notwithstanding the predicted demise of signature-based network monitoring, it is still part of the bedrock of security operations. Rulesets are fundamental to the efficacy of Network Intrusion Detection Systems (NIDS). Yet, they have rarely been studied in production environments. We partner with a Managed Security Service Provider (MSSP) to gain more insight into the evolution of rulesets, the alerts that they trigger and the incidents that get investigated. We analyze a combined ruleset - including both commercial and proprietary rules - that consists of 130 thousand rules and was used to monitor hundreds of networks. We find that these rulesets keep growing over time but there is almost no overlap among them in terms of detection options or what indicators of compromise they contain. The combined ruleset triggered more than 62 million alerts and led to 150 thousand incident investigations by SOC analysts, though the vast majority of rules never triggered a single alert. We find that just 0.5% of all rules are responsible for more than 80% of the alerts and incidents and only 1.2% of all alerts were deemed to merit closer investigation. Of all incidents, 16% were labeled as false positives and 9% carried significant risk to the client organization. Independently of the type of rule, updating rules is a minor activity. Most rules are never modified and only a fraction is deleted, except for periodic purges in some sets. Seven in-depth interviews with rule developers corroborate the patterns we found in our analysis. Finally, we identify several rule management practices that influence rule and ruleset efficacy, such as supplementing commercial rules with your own and making rules as specific as possible.
Original languageEnglish
Title of host publicationASIA CCS 2022 - Proceedings of the 2022 ACM Asia Conference on Computer and Communications Security
PublisherAssociation for Computing Machinery (ACM)
Pages799-813
Number of pages15
ISBN (Electronic)978-1-4503-9140-5
DOIs
Publication statusPublished - 2022
Event17th ACM ASIA Conference on Computer and Communications Security 2022, ASIA CCS 2022 - Virtual, Online, Japan
Duration: 30 May 20223 Jun 2022

Publication series

NameASIA CCS 2022 - Proceedings of the 2022 ACM Asia Conference on Computer and Communications Security

Conference

Conference17th ACM ASIA Conference on Computer and Communications Security 2022, ASIA CCS 2022
Country/TerritoryJapan
CityVirtual, Online
Period30/05/223/06/22

Keywords

  • alerts
  • intrusion detection
  • network security
  • nids
  • rules
  • soc

Fingerprint

Dive into the research topics of 'Ruling the Rules: Quantifying the Evolution of Rulesets, Alerts and Incidents in Network Intrusion Detection'. Together they form a unique fingerprint.

Cite this