Clonos: Consistent Causal Recovery for Highly-Available Streaming Dataflows

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

4 Citations (Scopus)
31 Downloads (Pure)


Stream processing lies in the backbone of modern businesses, being employed for mission critical applications such as real-time fraud detection, car-trip fare calculations, traffic management, and stock trading. Large-scale applications are executed by scale-out stream processing systems on thousands of long-lived operators, which are subject to failures. Recovering from failures fast and consistently are both top priorities, yet they are only partly satisfied by existing fault tolerance methods due to the strong assumptions these make. In particular, prior solutions fail to address consistency in the presence of nondeterminism, such as calls to external services, asynchronous timers and processing-time windows. This paper describes Clonos, a fault tolerance approach that achieves fast, local operator recovery with exactly-once guarantees and high availability by instantly switching to passive standby operators. Clonos enforces causally consistent recovery, including output deduplication, by tracking nondeterminism within the system through causal logging. To implement Clonos we re-engineered many of the internal subsystems of a state of the art stream processor. We evaluate Clonos' overhead and recovery on the Nexmark benchmark against Apache Flink. Clonos achieves instant recovery with negligible overhead and, unlike previous work, does not make assumptions on the deterministic nature of operators.

Original languageEnglish
Title of host publicationSIGMOD 2021 - Proceedings of the 2021 International Conference on Management of Data
PublisherAssociation for Computing Machinery (ACM)
Number of pages14
ISBN (Electronic)9781450383431
Publication statusPublished - 2021
Event2021 International Conference on Management of Data, SIGMOD 2021 - Virtual, Online, China
Duration: 20 Jun 202125 Jun 2021

Publication series

NameProceedings of the ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems


Conference2021 International Conference on Management of Data, SIGMOD 2021
CityVirtual, Online


  • cloud computing
  • consistency
  • exactly-once
  • fault-tolerance
  • high-availability
  • stream processing


Dive into the research topics of 'Clonos: Consistent Causal Recovery for Highly-Available Streaming Dataflows'. Together they form a unique fingerprint.

Cite this