Abstract
Although the cloud has reached a state of robustness, the burden of using its resources falls on the shoulders of programmers who struggle to keep up with ever-growing cloud infrastructure services and abstractions. As a result, state management, scaling, operation, and failure management of scalable cloud applications require disproportionately more effort than developing the applications' actual business logic. Our vision aims to raise the abstraction level for programming scalable cloud applications by compiling stateful entities – a programming model enabling imperative transactional programs authored in Python – into stateful streaming dataflows. We propose a compiler pipeline that analyzes the abstract syntax tree of stateful entities and transforms them into an intermediate representation based on stateful dataflow graphs. It then compiles that intermediate representation into different dataflow engines, leveraging their exactly-once message processing guarantees to prevent state or failure management primitives from "leaking" into the level of the programming model. Preliminary experiments with a proof of concept implementation show that despite program transformation and translation to dataflows, stateful entities can perform at sub-100ms latency even for transactional workloads.
Original language | English |
---|---|
Title of host publication | EDBT 2024 - 27th International Conference on Extending Database Technology |
Pages | 15-21 |
ISBN (Electronic) | 978-3-89318-091-2 |
Publication status | Published - 2024 |
Event | 27th International Conference on Extending Database Technology - Paestum, Italy Duration: 25 Mar 2024 → 28 Mar 2024 Conference number: 27 |
Conference
Conference | 27th International Conference on Extending Database Technology |
---|---|
Abbreviated title | EDBT 2024 |
Country/Territory | Italy |
City | Paestum |
Period | 25/03/24 → 28/03/24 |