Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineering

Z. Li, Agnia Sergeyuk, M. Izadi

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

Large Language Models are transforming software engineering, yet prompt management in practice remains ad hoc, hindering reliability, reuse, and integration into industrial workflows. We present Prompt-with-Me, a practical solution for structured prompt management embedded directly in the development environment. The system automatically classifies prompts using a four-dimensional taxonomy that encompasses intent, author role, software development lifecycle stage, and prompt type. To improve prompt reuse and quality, Prompt-with-Me suggests language refinements, masks sensitive information, and extracts reusable templates from a developer’s prompt library.Our taxonomy study of 1,108 real-world prompts demonstrates that modern LLMs can accurately classify software engineering prompts. Furthermore, our user study with 11 participants shows strong developer acceptance, with high usability (Mean SUS=73), low cognitive load (Mean NASA-TLX=21), and reported gains in prompt quality and efficiency through reduced repetitive effort. Lastly, we offer actionable insights for building the next generation of prompt management and maintenance tools for software engineering workflows.
Original languageEnglish
Title of host publicationProceedings of the 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE)
EditorsJ. Gurrola
Place of PublicationPiscataway
PublisherIEEE
Pages3346-3356
Number of pages11
ISBN (Electronic)979-8-3503-5733-2
ISBN (Print)979-8-3503-5734-9
DOIs
Publication statusPublished - 2025
Event2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) - Seoul, Korea, Republic of
Duration: 16 Nov 202520 Nov 2025

Conference

Conference2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE)
Country/TerritoryKorea, Republic of
CitySeoul
Period16/11/2520/11/25

Keywords

  • Large Language Models
  • Human Computer Interaction
  • Prompt Engineering
  • Software Engineering

Fingerprint

Dive into the research topics of 'Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineering'. Together they form a unique fingerprint.

Cite this