Green Runner: A Tool for Efficient Deep Learning Component Selection

Jai Kannan*, Scott Barnett, Luis Cruz, Anj Simmons, Taylan Selvi

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

For software that relies on machine-learned functionality, model selection is key to finding the right model for the task with desired performance characteristics. Evaluating a model requires developers to i) select from many models (e.g. the Hugging face model repository), ii) select evaluation metrics and training strategy, and iii) tailor trade-offs based on the problem domain. However, current evaluation approaches are either ad-hoc resulting in sub-optimal model selection or brute force leading to wasted compute. In this work, we present GreenRunner, a novel tool to automatically select and evaluate models based on the application scenario provided in natural language. We leverage the reasoning capabilities of large language models to propose a training strategy and extract desired trade-offs from a problem description. GreenRunner features a resource-efficient experimentation engine that integrates constraints and trade-offs based on the problem into the model selection process. Our preliminary evaluation demonstrates that GreenRunner is both efficient and accurate compared to ad-hoc evaluations and brute force. This work presents an important step toward energy-efficient tools to help reduce the environmental impact caused by the growing demand for software with machine-learned functionality. Our tool is available at Figshare GreenRunner.

Original languageEnglish
Title of host publicationProceedings - 2024 IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI, CAIN 2024
PublisherAssociation for Computing Machinery (ACM)
Pages112-117
Number of pages6
ISBN (Electronic)9798400705915
DOIs
Publication statusPublished - 2024
Event3rd International Conference on AI Engineering, CAIN 2024, co-located with the 46th International Conference on Software Engineering, ICSE 2024 - Lisbon, Portugal
Duration: 14 Apr 202415 Apr 2024

Publication series

NameProceedings - 2024 IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI, CAIN 2024

Conference

Conference3rd International Conference on AI Engineering, CAIN 2024, co-located with the 46th International Conference on Software Engineering, ICSE 2024
Country/TerritoryPortugal
CityLisbon
Period14/04/2415/04/24

Keywords

  • component selection
  • green-AI
  • large language model

Fingerprint

Dive into the research topics of 'Green Runner: A Tool for Efficient Deep Learning Component Selection'. Together they form a unique fingerprint.

Cite this