Pythia: A customizable hardware prefetching framework using online reinforcement learning

Rahul Bera, Konstantinos Kanellopoulos, Anant V. Nori, Taha Shahroodi, Sreenivas Subramoney, Onur Mutlu

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

38 Citations (Scopus)

Abstract

Past research has proposed numerous hardware prefetching techniques, most of which rely on exploiting one specific type of program context information (e.g., program counter, cacheline address, or delta between cacheline addresses) to predict future memory accesses. These techniques either completely neglect a prefetcher's undesirable effects (e.g., memory bandwidth usage) on the overall system, or incorporate system-level feedback as an afterthought to a system-unaware prefetch algorithm.We showthat prior prefetchers often lose their performance benefit over a wide range of workloads and system configurations due to their inherent inability to take multiple different types of program context and system-level feedback information into account while prefetching. In this paper, we make a case for designing a holistic prefetch algorithm that learns to prefetch using multiple different types of program context and system-level feedback information inherent to its design. To this end, we propose Pythia, which formulates the prefetcher as a reinforcement learning agent. For every demand request, Pythia observes multiple different types of program context information to make a prefetch decision. For every prefetch decision, Pythia receives a numerical reward that evaluates prefetch quality under the current memory bandwidth usage. Pythia uses this reward to reinforce the correlation between program context information and prefetch decision to generate highly accurate, timely, and systemaware prefetch requests in the future. Our extensive evaluations using simulation and hardware synthesis show that Pythia outperforms two state-of-the-art prefetchers (MLOP and Bingo) by 3.4% and 3.8% in single-core, 7.7% and 9.6% in twelve-core, and 16.9% and 20.2% in bandwidth-constrained core configurations, while incurring only 1.03% area overhead over a desktop-class processor and no software changes in workloads. The source code of Pythia can be freely downloaded from https://github.com/CMU-SAFARI/Pythia.

Original languageEnglish
Title of host publicationMICRO 2021 - 54th Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings
PublisherIEEE
Pages1121-1137
Number of pages17
ISBN (Electronic)9781450385572
DOIs
Publication statusPublished - 2021
Event54th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2021 - Virtual, Online, Greece
Duration: 18 Oct 202122 Oct 2021

Conference

Conference54th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2021
Country/TerritoryGreece
CityVirtual, Online
Period18/10/2122/10/21

Fingerprint

Dive into the research topics of 'Pythia: A customizable hardware prefetching framework using online reinforcement learning'. Together they form a unique fingerprint.

Cite this