Parallelization of variable rate decompression through metadata

Lennart Noordsij, Steven Van Der Vlugt, Mohamed A. Bamakhrama, Zaid Al-Ars, Peter Lindstrom

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

5 Citations (Scopus)
84 Downloads (Pure)

Abstract

Data movement has long been identified as the biggest challenge facing modern computer systems' designers. To tackle this challenge, many novel data compression algorithms have been developed. Often variable rate compression algorithms are favored over fixed rate. However, variable rate decompression is difficult to parallelize. Most existing algorithms adopt a single parallelization strategy suited for a particular HW platform. Such an approach fails to harness the parallelism found in diverse modern HW architectures. We propose a parallelization method for tiled variable rate compression algorithms that consists of multiple strategies that can be applied interchangeably. This allows an algorithm to apply the strategy most suitable for a specific HW platform. Our strategies are based on generating metadata during encoding, which is used to parallelize the decoding process. To demonstrate the effectiveness of our strategies, we implement them in a state-of-the-art compression algorithm called ZFP. We show that the strategies suited for multicore CPUs are different from the ones suited for GPUs. On a CPU, we achieve a near optimal decoding speedup and an overhead size which is consistently less than 0.04% of the compressed data size. On a GPU, we achieve average decoding rates of up to 100 GiB/s. Our strategies allow the user to make a trade-off between decoding throughput and metadata size overhead.

Original languageEnglish
Title of host publicationProceedings - 2020 28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2020
EditorsMasoud Daneshtalab, Leporati Francesco, Mikael Sjödin
PublisherIEEE
Pages245-252
Number of pages8
ISBN (Electronic)9781728165820
DOIs
Publication statusPublished - 2020
Event28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2020 - Vasteras, Sweden
Duration: 11 Mar 202013 Mar 2020

Conference

Conference28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2020
Country/TerritorySweden
CityVasteras
Period11/03/2013/03/20

Fingerprint

Dive into the research topics of 'Parallelization of variable rate decompression through metadata'. Together they form a unique fingerprint.

Cite this