A Systematic Comparison of Search-Based Approaches for LDA Hyperparameter Tuning

Research output: Contribution to journalArticleScientificpeer-review

4 Downloads (Pure)

Abstract

Context: Latent Dirichlet Allocation (LDA) has been successfully used in the literature to extract topics from software documents and support developers in various software engineering tasks. While LDA has been mostly used with default settings, previous studies showed that default hyperparameter values generate sub-optimal topics from software documents. Objective: Recent studies applied meta-heuristic search (mostly evolutionary algorithms) to configure LDA in an unsupervised and automated fashion. However, previous work advocated for different meta-heuristics and surrogate metrics to optimize. The objective of this paper is to shed light on the influence of these two factors when tuning LDA for SE tasks. Method: We empirically evaluated and compared seven state-of-the-art meta-heuristics and three alternative surrogate metrics (i.e., fitness functions) to solve the problem of identifying duplicate bug reports with LDA. The benchmark consists of ten real-world and open-source projects from the Bench4BL} dataset. Results: Our results indicate that (1) meta-heuristics are mostly comparable to one another (except for random search and CMA-ES), and (2) the choice of the surrogate metric impacts the quality of the generated topics and the tuning overhead. Furthermore, calibrating LDA helps identify twice as many duplicates than untuned LDA when inspecting the top five past similar reports. Conclusion: No meta-heuristic and/or fitness function outperforms all the others, as advocated in prior studies. However, we can make recommendations for some combinations of meta-heuristics and fitness functions over others for practical use. Future work should focus on improving the surrogate metrics used to calibrate/tune LDA in an unsupervised fashion
Original languageEnglish
Article number106411
Number of pages20
JournalInformation and Software Technology
Volume130
DOIs
Publication statusPublished - 2021

Keywords

  • Duplicate bug report
  • Hyperparameter optimization
  • Latent dirichlet allocation
  • Metaheuristic search
  • Search-based software engineering
  • Topic modeling

Fingerprint Dive into the research topics of 'A Systematic Comparison of Search-Based Approaches for LDA Hyperparameter Tuning'. Together they form a unique fingerprint.

Cite this