SparkRA: Enabling big data scalability for the GATK RNA-seq pipeline with apache spark

Zaid Al-Ars*, Saiyi Wang, Hamid Mushtaq

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

5 Citations (Scopus)
60 Downloads (Pure)


The rapid proliferation of low-cost RNA-seq data has resulted in a growing interest in RNA analysis techniques for various applications, ranging from identifying genotype–phenotype relationships to validating discoveries of other analysis results. However, many practical applications in this field are limited by the available computational resources and associated long computing time needed to perform the analysis. GATK has a popular best practices pipeline specifically designed for variant calling RNA-seq analysis. Some tools in this pipeline are not optimized to scale the analysis to multiple processors or compute nodes efficiently, thereby limiting their ability to process large datasets. In this paper, we present SparkRA, an Apache Spark based pipeline to efficiently scale up the GATK RNA-seq variant calling pipeline on multiple cores in one node or in a large cluster. On a single node with 20 hyper-threaded cores, the original pipeline runs for more than 5 h to process a dataset of 32 GB. In contrast, SparkRA is able to reduce the overall computation time of the pipeline on the same single node by about 4×, reducing the computation time down to 1.3 h. On a cluster with 16 nodes (each with eight single-threaded cores), SparkRA is able to further reduce this computation time by 7.7× compared to a single node. Compared to other scalable state-of-the-art solutions, SparkRA is 1.2× faster while achieving the same accuracy of the results.

Original languageEnglish
Article number53
Issue number1
Publication statusPublished - 1 Jan 2020


  • Apache Spark
  • Computation time
  • GATK variant calling
  • RNA-seq
  • Scalability


Dive into the research topics of 'SparkRA: Enabling big data scalability for the GATK RNA-seq pipeline with apache spark'. Together they form a unique fingerprint.

Cite this