Operational stream processing: Towards scalable and consistent event-driven applications

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

7 Citations (Scopus)
420 Downloads (Pure)


In the last decade we are witnessing a widespread adoption of architectural styles such as microservices, for building event-driven software applications and deploying them in cloud infrastructures. Such services favor the separation of a database into independent silos of data, each of which is owned entirely by a single service. As a result, traditional oltp systems no longer fit the architectural picture and developers often turn to ad-hoc solutions that rarely support acid transaction consistency. At the same time, we are witnessing the gradual maturation of distributed streaming dataflow systems. These systems nowadays have departed from the mere analysis of streaming windows and complex-event processing, employing sophisticated methods for managing state, keeping it consistent, and ensuring exactly-once processing guarantees in the presence of failures. The goal of this paper is threefold. First, we illustrate the requirements of stateful software services in terms of consistency and scalability. Second, we present how well existing solutions meet those requirements. Finally, we outline a set of challenging problems and propose research directions for enabling event-driven applications to be developed on top of streaming dataflow systems. We strongly believe that streaming dataflows can have a central place in service-oriented architectures, taking over the execution of acid transactions, ensuring message delivery and processing, in order to perform scalable execution of services.

Original languageEnglish
Title of host publicationAdvances in Database Technology - EDBT 2019
Subtitle of host publication22nd International Conference on Extending Database Technology, Proceedings
EditorsBerthold Reinwald, Helena Galhardas, Irini Fundulaki, Melanie Herschel, Zoi Kaoudi, Carsten Binnig
Number of pages4
ISBN (Electronic)978-3-89318-081-3
Publication statusPublished - 2019
Event22nd International Conference on Extending Database Technology, EDBT 2019 - Lisbon, Portugal
Duration: 26 Mar 201929 Mar 2019


Conference22nd International Conference on Extending Database Technology, EDBT 2019


Dive into the research topics of 'Operational stream processing: Towards scalable and consistent event-driven applications'. Together they form a unique fingerprint.

Cite this