Supporting Columnar In-memory Formats on FPGA: The Hardware Design of Fletcher for Apache Arrow

Johan Peltenburg, Jeroen van Straten, Matthijs Brobbel, H. Peter Hofstee, Zaid Al-Ars

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

10 Citations (Scopus)
172 Downloads (Pure)

Abstract

As a columnar in-memory format, Apache Arrow has seen increased interest from the data analytics community. Fletcher is a framework that generates hardware interfaces based on this format, to be used in FPGA accelerators. This allows efficient integration of FPGA accelerators with various high-level software languages, while providing an easy-to-use hardware interface for the FPGA developer. The abstract descriptions of data sets stored in the Arrow format, that form the input of the interface generation step, can be complex. To generate efficient interfaces from it is challenging. In this paper, we introduce the hardware components of Fletcher that help solve this challenge. These components allow FPGA developers to express access to complex Arrow data records through row indices of tabular data sets, rather than through byte addresses. The data records are delivered as streams of the same abstract types as found in the data set, rather than as memory bus words. The generated interfaces allow for full system bandwidth to be utilized and have a low area profile. All components are open sourced and available for other researchers and developers to use in their projects.
Original languageEnglish
Title of host publicationApplied Reconfigurable Computing
Subtitle of host publication15th International Symposium, ARC 2019, Proceedings
EditorsChristian Hochberger, Andreas Koch, Pedro Diniz, Roger Woods, Brent Nelson
Place of PublicationCham
PublisherSpringer
Pages32-47
Number of pages16
ISBN (Electronic)978-3-030-17227-5
ISBN (Print)978-3-030-17226-8
DOIs
Publication statusPublished - 2019
Event15th International Symposium on Applied Reconfigurable Computing, ARC 2019 - Darmstadt, Germany
Duration: 9 Apr 201911 Apr 2019

Publication series

NameLecture Notes in Computer Science
Volume11444 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th International Symposium on Applied Reconfigurable Computing, ARC 2019
Country/TerritoryGermany
CityDarmstadt
Period9/04/1911/04/19

Keywords

  • Apache Arrow
  • Fletcher
  • FPGA

Fingerprint

Dive into the research topics of 'Supporting Columnar In-memory Formats on FPGA: The Hardware Design of Fletcher for Apache Arrow'. Together they form a unique fingerprint.

Cite this