Stateful Entities: Object-oriented Cloud Applications as Distributed Dataflows

K. Psarakis, W.D. Zorgdrager, M. Fragkoulis, Guido Salvaneschi, A Katsifodimos

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

Although the cloud has reached a state of robustness, the burden of using its resources falls on the shoulders of programmers who struggle to keep up with ever-growing cloud infrastructure services and abstractions. As a result, state management, scaling, operation, and failure management of scalable cloud applications require disproportionately more effort than developing the applications' actual business logic. Our vision aims to raise the abstraction level for programming scalable cloud applications by compiling stateful entities – a programming model enabling imperative transactional programs authored in Python – into stateful streaming dataflows. We propose a compiler pipeline that analyzes the abstract syntax tree of stateful entities and transforms them into an intermediate representation based on stateful dataflow graphs. It then compiles that intermediate representation into different dataflow engines, leveraging their exactly-once message processing guarantees to prevent state or failure management primitives from "leaking" into the level of the programming model. Preliminary experiments with a proof of concept implementation show that despite program transformation and translation to dataflows, stateful entities can perform at sub-100ms latency even for transactional workloads.
Original languageEnglish
Title of host publicationEDBT 2024 - 27th International Conference on Extending Database Technology
Pages15-21
ISBN (Electronic)978-3-89318-091-2
Publication statusPublished - 2024
Event27th International Conference on Extending Database Technology - Paestum, Italy
Duration: 25 Mar 202428 Mar 2024
Conference number: 27

Conference

Conference27th International Conference on Extending Database Technology
Abbreviated titleEDBT 2024
Country/TerritoryItaly
CityPaestum
Period25/03/2428/03/24

Fingerprint

Dive into the research topics of 'Stateful Entities: Object-oriented Cloud Applications as Distributed Dataflows'. Together they form a unique fingerprint.

Cite this