Gromit: Benchmarking the Performance and Scalability of Blockchain Systems

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

1 Citation (Scopus)
14 Downloads (Pure)

Abstract

The growing number of implementations of blockchain systems stands in stark contrast with still limited research on a systematic comparison of performance characteristics of these solutions. Such research is crucial for evaluating fundamental trade-offs introduced by novel consensus protocols and their implementations. These performance limitations are commonly analyzed with ad-hoc benchmarking frameworks focused on the consensus algorithm of blockchain systems. However, comparative evaluations of design choices require macro-benchmarks for uniform and comprehensive performance evaluations of blockchains at the system level rather than performance metrics of isolated components. To address this research gap, we implement Gromit, a generic framework for analyzing blockchain systems. Gromit treats each system under test as a transaction fabric where clients issue transactions to validators. We use Gromit to conduct the largest blockchain study to date, involving seven representative systems with varying consensus models. We determine the peak performance of these systems with a synthetic workload in terms of transaction throughput and scalability and show that transaction throughput does not scale with the number of validators. We explore how robust the subjected systems are against network delays and reveal that the performance of permissoned blockchain is highly sensitive to network conditions.
Original languageEnglish
Title of host publicationProceedings of the 2022 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS)
EditorsL. O'Conner
Place of PublicationPiscataway
PublisherIEEE
Pages56-63
Number of pages8
ISBN (Electronic)978-1-6654-9172-3
ISBN (Print)978-1-6654-9173-0
DOIs
Publication statusPublished - 2022
Event2022 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS) - Newark, United States
Duration: 15 Aug 202218 Aug 2022

Publication series

NameProceedings - 4th IEEE International Conference on Decentralized Applications and Infrastructures, DAPPS 2022

Conference

Conference2022 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPS)
Country/TerritoryUnited States
CityNewark
Period15/08/2218/08/22

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Benchmark
  • blockchain performance
  • reproducibility
  • stress testing

Fingerprint

Dive into the research topics of 'Gromit: Benchmarking the Performance and Scalability of Blockchain Systems'. Together they form a unique fingerprint.

Cite this