TY - JOUR
T1 - Hazelcast jet
T2 - 47th International Conference on Very Large Data Bases, VLDB 2021
AU - Gencer, Can
AU - Topolnik, Marko
AU - Ďurina, Viliam
AU - Demirci, Emin
AU - Kahveci, Ensar B.
AU - Gürbüz, Ali
AU - Lukáš, Ondřej
AU - Fragkoulis, Marios
AU - Katsifodimos, Asterios
AU - More Authors, null
PY - 2021
Y1 - 2021
N2 - Jet is an open-source, high-performance, distributed stream processor built at Hazelcast during the last five years. Jet was engineered with millisecond latency on the 99.99th percentile as its primary design goal. Originally Jet’s purpose was to be an execution engine that performs complex business logic on top of streams generated by Hazelcast’s In-memory Data Grid (IMDG): a set of in-memory, partitioned and replicated data structures. With time, Jet evolved into a full-fledged, scale-out stream processor that can handle out-of-order streams and provide exactly-once processing guarantees. Jet’s end-to-end latency lies in the order of milliseconds, and its throughput in the order of millions of events per CPU-core. This paper presents the main design decisions we made in order to maximize the performance per CPU-core, alongside lessons learned, and an empirical performance evaluation.
AB - Jet is an open-source, high-performance, distributed stream processor built at Hazelcast during the last five years. Jet was engineered with millisecond latency on the 99.99th percentile as its primary design goal. Originally Jet’s purpose was to be an execution engine that performs complex business logic on top of streams generated by Hazelcast’s In-memory Data Grid (IMDG): a set of in-memory, partitioned and replicated data structures. With time, Jet evolved into a full-fledged, scale-out stream processor that can handle out-of-order streams and provide exactly-once processing guarantees. Jet’s end-to-end latency lies in the order of milliseconds, and its throughput in the order of millions of events per CPU-core. This paper presents the main design decisions we made in order to maximize the performance per CPU-core, alongside lessons learned, and an empirical performance evaluation.
UR - http://www.scopus.com/inward/record.url?scp=85110289668&partnerID=8YFLogxK
U2 - 10.14778/3476311.3476387
DO - 10.14778/3476311.3476387
M3 - Conference article
AN - SCOPUS:85110289668
SN - 2150-8097
VL - 14
SP - 3110
EP - 3121
JO - Proceedings of the VLDB Endowment
JF - Proceedings of the VLDB Endowment
IS - 12
Y2 - 16 August 2021 through 20 August 2021
ER -