Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries

Ali Al-Kaswan, Toufique Ahmed, Maliheh Izadi, Anand Ashok Sawant, Premkumar Devanbu, Arie van Deursen

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

3 Citations (Scopus)
36 Downloads (Pure)

Abstract

Binary reverse engineering is used to understand and analyse programs for which the source code is unavailable. Decompilers can help, transforming opaque binaries into a more readable source code-like representation. Still, reverse engineering is difficult and costly, involving considering effort in labelling code with helpful summaries. While the automated summarisation of decompiled code can help reverse engineers understand and analyse binaries, current work mainly focuses on summarising source code, and no suitable dataset exists for this task. In this work, we extend large pre-trained language models of source code to summarise de-compiled binary functions. Further-more, we investigate the impact of input and data properties on the performance of such models. Our approach consists of two main components; the data and the model. We first build CAPYBARA, a dataset of 214K decompiled function-documentation pairs across various compiler optimisations. We extend CAPYBARA further by removing identifiers, and deduplicating the data. Next, we fine-tune the CodeT5 base model with CAPYBARA to create BinT5. BinT5 achieves the state-of-the-art BLEU-4 score of 60.83, 58.82 and, 44.21 for summarising source, decompiled, and obfuscated decompiled code, respectively. This indicates that these models can be extended to decompiled binaries successfully. Finally, we found that the performance of BinT5 is not heavily dependent on the dataset size and compiler optimisation level. We recommend future research to further investigate transferring knowledge when working with less expressive input formats such as stripped binaries.

Original languageEnglish
Title of host publicationProceedings of the 30th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)
EditorsCristina Ceballos
Place of PublicationPiscataway
PublisherIEEE
Pages260-271
Number of pages12
ISBN (Electronic)978-1-6654-5278-6
ISBN (Print)978-1-6654-5279-3
DOIs
Publication statusPublished - 2023
Event2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) - Taipa, Macao
Duration: 21 Mar 202324 Mar 2023

Conference

Conference2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)
Country/TerritoryMacao
City Taipa
Period21/03/2324/03/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Decompilation
  • Binary
  • Reverse Engineering
  • Summarization
  • Deep Learning
  • Pre-trained Language Models
  • CodeT5
  • Transformers

Fingerprint

Dive into the research topics of 'Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries'. Together they form a unique fingerprint.

Cite this