The growing number of implementations of blockchain systems stands in stark contrast with still limited research on a systematic comparison of performance characteristics of these solutions. Such research is crucial for evaluating fundamental trade-offs introduced by novel consensus protocols and their implementations. These performance limitations are commonly analyzed with ad-hoc benchmarking frameworks focused on the consensus algorithm of blockchain systems. However, comparative evaluations of design choices require macro-benchmarks for uniform and comprehensive performance evaluations of blockchains at the system level rather than performance metrics of isolated components. To address this research gap, we implement Gromit, a generic framework for analyzing blockchain systems. Gromit treats each system under test as a transaction fabric where clients issue transactions to validators. We use Gromit to conduct the largest blockchain study to date, involving seven representative systems with varying consensus models. We determine the peak performance of these systems with a synthetic workload in terms of transaction throughput and scalability and show that transaction throughput does not scale with the number of validators. We explore how robust the subjected systems are against network delays and reveal that the performance of permissoned blockchain is highly sensitive to network conditions.
|Title of host publication||Proceedings of the 2022 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS)|
|Place of Publication||Piscataway|
|Number of pages||8|
|Publication status||Published - 2022|
|Event||2022 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS) - Newark, United States|
Duration: 15 Aug 2022 → 18 Aug 2022
|Name||Proceedings - 4th IEEE International Conference on Decentralized Applications and Infrastructures, DAPPS 2022|
|Conference||2022 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS)|
|Period||15/08/22 → 18/08/22|
Bibliographical noteGreen Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
- blockchain performance
- stress testing