WP7D3: User tests preparation and results (COMESI)

WAGA Bouwman, GA de Reuver, S Nikou

Research output: Book/ReportReportProfessional


In this report we discuss the experimental set up, methodology and results of a laboratory experiment carried out within the scope of the Celtic-Comesi project. The experiment aims to get user feedback on the user experience and technical functionality of two prototype applications developed in the project: Content anywhere and Social communication on TV. The experiment was carried out in Delft in September 2011. Test subjects were 53 students from Delft University of Technology. The experiment shows that the technology still needs to be developed further, especially regarding the stability of the user clients and servers. Furthermore, the user interface design requires further attention for both applications as test subjects required a lot of help to utilize the applications. A more intuitive interface is necessary will the Comesi applications live up to the positive expectations of the subjects. Data extracted from the experiment show that the Comesi features in the applications were generally considered useful. Especially presence and availability features were considered to be even more useful after testing the applications. Respondents became more positive about the usefulness of Comesi features after they tried out the applications, which underline the relevance of prototyping and user testing. On the other hand, user willingness to pay is low, and users would rather adopt the services from web companies than from operators. However, the reader should be aware that the subjects are students with low budgets, and that Dutch people are very critical about pricing in general. Operators looking to roll out the Comesi services should invest in user trust as this the only construct that significantly impacts user decision to adopt services from operators rather than from web companies. The experiment results also indicate that the Content anywhere application was evaluated much more positively than the Social communication on TV application. Key issues that affect the user¿s intention to adopt the applications include the perceived usefulness, ease of use and to a lesser extent, trust in the operators. A number of subjects indicated that they would really like the TV application, assuming that the technical errors that did occurred during the experiment, would be mended. The TV application in a sense was a new experience for many of them. Convergence of television and telecom technology clearly can strengthen Telcos¿ position towards web companies. Letting users experience the applications does add value, as their perception on specific Comesi functions like presence on TV and availability features on the phone and web increase after they try out applications. Moreover, subjects clearly indicated that they would have liked to explore the functionalities in more detail. The results are partly confirming, partly conflicting with the findings of the earlier conjoint analysis in WP7D2. While switching media and switching devices was hardly found useful in the conjoint analysis, these features score rather high in this experiment, not only after trying out the applications but even beforehand. A possible explanation might be the nature of the group of subjects: employees of telecom operators and ICT experts dominated the conjoint sample, while students are subject in this experiment.
Original languageEnglish
Place of PublicationDelft
PublisherCeltic COMESI
Number of pages80
Publication statusPublished - 2011

Publication series

PublisherCeltic COMESI


  • Geen BTA classificatie


Dive into the research topics of 'WP7D3: User tests preparation and results (COMESI)'. Together they form a unique fingerprint.

Cite this