Human-AI experience in integrated development environments: a systematic literature review

Agnia Sergeyuk*, Ilya Zakharov, Ekaterina Koshchenko, Maliheh Izadi

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

The integration of Artificial Intelligence (AI) into Integrated Development Environments (IDEs) is reshaping software development, fundamentally altering how developers interact with their tools. This shift marks the emergence of Human-AI Experience in Integrated Development Environment (in-IDE HAX), a field that explores the evolving dynamics of Human-Computer Interaction in AI-assisted coding environments. Despite rapid adoption, research on in-IDE HAX remains fragmented, which highlights the need for a unified overview of current practices, challenges, and opportunities. To provide a structured overview of existing research, we conduct a systematic literature review of 90 studies, summarizing current findings and outlining areas for further investigation. We organize key insights from reviewed studies into three aspects: Impact, Design, and Quality of AI-based systems inside IDEs. Impact findings show that AI-assisted coding enhances developer productivity but also introduces challenges, such as verification overhead and over-reliance. Design studies show that effective interfaces surface context, provide explanations and transparency of suggestion, and support user control. Quality studies document risks in correctness, maintainability, and security. For future research, priorities include productivity studies, design of assistance, and audit of AI-generated code. The agenda calls for larger and longer evaluations, stronger audit and verification assets, broader coverage across the software life cycle, and adaptive assistance under user control.
Original languageEnglish
Article number55
Number of pages35
JournalEmpirical Software Engineering
Volume31
Issue number3
DOIs
Publication statusPublished - 2026

Bibliographical note

Green Open Access added to TU Delft Institutional Repository as part of the Taverne amendment. More information about this copyright law amendment can be found at https://www.openaccess.nl. Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Artificial intelligence
  • Human-computer interaction
  • Integrated development environment
  • Programming
  • User experience
  • User studies

Fingerprint

Dive into the research topics of 'Human-AI experience in integrated development environments: a systematic literature review'. Together they form a unique fingerprint.

Cite this