SparkGA: A Spark Framework for Cost Effective, Fast and Accurate DNA Analysis at Scale

Hamid Mushtaq, Frank Liu, Carlos Costa, Gang Liu, Peter Hofstee, Zaid Al-Ars

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

25 Citations (Scopus)

Abstract

In recent years, the cost of NGS (Next Generation Sequencing) technology has dramatically reduced, making it a viable method for
diagnosing genetic diseases. The large amount of data generated by NGS technology, usually in the order of hundreds of gigabytes per experiment, have to be analyzed quickly to generate meaningful variant results. The GATK best practices pipeline from the Broad
Institute is one of the most popular computational pipelines for DNA analysis. Many components of the GATK pipeline are not very
parallelizable though. In this paper, we present SparkGA, a parallel implementation of a DNA analysis pipeline based on the big data
Apache Spark framework. This implementation is highly scalable and capable of parallelizing computation by utilizing data-level
parallelism as well as load balancing techniques. In order to reduce the analysis cost, SparkGA can run on nodes with as little memory as 16GB. For whole genome sequencing experiments, we show that the runtime can be reduced to about 1.5 hours on a 20-node cluster with an accuracy of up to 99.9981%. Moreover, SparkGA is about 71% faster than other state-of-the-art solutions while also being more accurate. The source code of SparkGA is publicly available at ttps://github.com/HamidMushtaq/SparkGA1.git.
Original languageEnglish
Title of host publicationACM-BCB '17 Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology,and Health Informatics
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Pages148-157
Number of pages10
ISBN (Print)978-1-4503-4722-8
DOIs
Publication statusPublished - 2017
Event8th ACM Conference on Bioinformatics, Computational Biology and Health Informatics (ACM-BCB 2017) - Boston, United States
Duration: 20 Aug 201723 Aug 2017

Conference

Conference8th ACM Conference on Bioinformatics, Computational Biology and Health Informatics (ACM-BCB 2017)
Country/TerritoryUnited States
CityBoston
Period20/08/1723/08/17

Fingerprint

Dive into the research topics of 'SparkGA: A Spark Framework for Cost Effective, Fast and Accurate DNA Analysis at Scale'. Together they form a unique fingerprint.

Cite this