Server-Side Experimentation: Common challenges from 12 organisations in The Netherlands

Background: experimentation is widely adopted within industry. Many large organisations have invested in their own infrastructure to be able to run experiments server-side. Objective: we wanted to investigate why organisations switch to server-side experimen- tation and which challenges they encounter in doing so. Method: a qualitative virtual discussion based on a survey was conducted. Twelve Dutch organisations, represented by twelve interviewees, participated in the study. Results: organisations switch to server- side experimentation to run higher quality, more advanced and cheaper experiments. Challenges organisations face are: a shortage of development resources, no standardized process and a lack of a culture of experimentation. Conclusions: this is the first study that explores why organisations from The Netherlands in transition to continuous experimentation choose for investing in server-side experimentation infrastructure, and the challenges they encounter.