FPGA Acceleration of Zstd Compression Algorithm

Jianyu Chen, Maurice Daverveldt, Zaid Al-Ars

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

4 Citations (Scopus)
447 Downloads (Pure)


With the continued increase in the amount of big data generated and stored in various application domains, such as high-frequency trading, compression techniques are becoming ever more important to reduce the requirements on communication bandwidth and storage capacity. Zstandard (Zstd) is emerging as an important compression algorithm for big data sets capable of achieving a good compression ratio but with a higher speed than comparable algorithms. In this paper, we introduce the architecture of a new hardware compression kernel for Zstd that allows the algorithm to be used for real-time compression of big data streams. In addition, we optimize the proposed architecture for the specific use case of streaming high-frequency trading data. The optimized kernel is implemented on a Xilinx Alveo U200 board. Our optimized implementation allows us to fit ten kernel blocks on one board, which is able to achieve a compression throughput of about 8.6GB/s and compression ratio of about 23.6%. The hardware implementation is open source and publicly available at https://github.com/ChenJianyunp/Hardware-Zstd-Compression-Unit.
Original languageEnglish
Title of host publication2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
EditorsL. O'Conner
Place of PublicationPiscataway
Number of pages4
ISBN (Electronic)978-1-6654-3577-2
ISBN (Print)978-1-6654-1192-9
Publication statusPublished - 2021
Event2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) - Virtually at Portland, United States
Duration: 17 Jun 202121 Jun 2021


Workshop2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
Country/TerritoryUnited States
CityVirtually at Portland

Bibliographical note

Accepted author manuscript


  • FPGA
  • Zstd
  • compression


Dive into the research topics of 'FPGA Acceleration of Zstd Compression Algorithm'. Together they form a unique fingerprint.

Cite this