A Reference Architecture for Datacenter Scheduling: Design, Validation, and Experiments

Georgios Andreadis, Laurens Versluis, Fabian Mastenbroek, Alexandru Iosup

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

2 Citations (Scopus)

Abstract

Datacenters act as cloud-infrastructure to stakeholders across industry, government, and academia. To meet growing demand yet operate efficiently, datacenter operators employ increasingly more sophisticated scheduling systems, mechanisms, and policies. Although many scheduling techniques already exist, relatively little research has gone into the abstraction of the scheduling process itself, hampering design, tuning, and comparison of existing techniques. In this work, we propose a reference architecture for datacenter schedulers. The architecture follows five design principles: components with clearly distinct responsibilities, grouping of related components where possible, separation of mechanism from policy, scheduling as complex workflow, and hierarchical multi-scheduler structure. To demonstrate the validity of the reference architecture, we map to it state-of-the-art datacenter schedulers. We find scheduler-stages are commonly underspecified in peer-reviewed publications. Through trace-based simulation and real-world experiments, we show underspecification of scheduler-stages can lead to significant variations in performance.

Original languageEnglish
Title of host publicationSC18: International Conference for High Performance Computing, Networking, Storage and Analysis
Subtitle of host publicationProceedings
Place of PublicationPiscataway
PublisherIEEE
Pages478-492
Number of pages15
ISBN (Electronic)978-1-5386-8384-2
ISBN (Print)978-1-5386-8385-9
DOIs
Publication statusPublished - 2019
Event2018 International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018 - Dallas, United States
Duration: 11 Nov 201816 Nov 2018

Conference

Conference2018 International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018
CountryUnited States
CityDallas
Period11/11/1816/11/18

Keywords

  • Datacenter
  • Reference Architecture
  • Scheduling

Fingerprint Dive into the research topics of 'A Reference Architecture for Datacenter Scheduling: Design, Validation, and Experiments'. Together they form a unique fingerprint.

  • Cite this

    Andreadis, G., Versluis, L., Mastenbroek, F., & Iosup, A. (2019). A Reference Architecture for Datacenter Scheduling: Design, Validation, and Experiments. In SC18: International Conference for High Performance Computing, Networking, Storage and Analysis: Proceedings (pp. 478-492). [8665816] IEEE. https://doi.org/10.1109/SC.2018.00040