Data-Enhanced Design: Engaging Designers in Exploratory Sensemaking with Multimodal Data

Katerina Gorkovenko, Adam Jenkins, Kami Vaniea, Dave Murray-Rust*

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

18 Downloads (Pure)


Research in the wild can reveal human behaviors, contexts, and needs around products that are difficult to observe in the lab. Telemetry data from the use of physical products can help facilitate in the wild research, in particular by suggesting hypotheses that can be explored through machine learning models. This paper explores ways for designers without strong data skills to engage with multimodal data to develop a contextual understanding of product use. This study is framed around a lightweight version of a data enhanced design research process where multimodal telemetry data was captured by a GoPro camera attached to a bicycle. This was combined with the video data and conversation with the rider to carry out an exploratory sensemaking process and generate design research questions that could potentially be addressed through data capture, annotation, and machine learning. We identify a range of ways that designers could make use of the data for ideation and developing context through annotating and exploring the data. Participants used data and annotation practices to connect the micro and macro, spot interesting moments, and frame questions around an unfamiliar problem. The work follows the designers’ questions, methods, and explorations, both immediate concerns and speculations about working at larger scales with machine learning models. This points to the possibility of tools that help designers to engage with machine learning, not just for optimization and refinement, but for creative ideation in the early stages of design processes.

Original languageEnglish
Pages (from-to)1-23
Number of pages23
JournalInternational Journal of Design
Issue number3
Publication statusPublished - 2023


This research was funded by the UK Research Councils / EPSRC under the New Industrial Systems theme through the Chatty Factories project (EP/R021031/1). Many thanks to all of our participants for joining in with research despite the pandemic, and to the Design Informatics community in Edinburgh for being a fantastic place to do interesting research.


  • Cycling
  • Design Research
  • GoPro Data
  • Multimodal Data Annotation


Dive into the research topics of 'Data-Enhanced Design: Engaging Designers in Exploratory Sensemaking with Multimodal Data'. Together they form a unique fingerprint.

Cite this