Winning Data: Designing and testing a game to change civil servants' attitudes towards open governmental data provision

Research output: ThesisDissertation (TU Delft)

11 Downloads (Pure)

Abstract

Data is needed for a government to function, and civil servants generate data that can be opened. However, this data is not always publicly available. Governments open their data to meet societal needs to increase transparency, accountability, stimulate participation and innovation. The opening of governmental data can be seen as a source of uncertainty for public servants, or it can even be legally prohibited, depending on how the regulation is interpreted. For instance, open data might be experienced as a burden or not easy to practice, whereas the opening might create societal relevance. This research focuses on overcoming behavioral barriers for civil servants to manage data release at the individual level by using a serious game. Open data relates to any data produced by any device or person, which is publicly shared for free or at a minimal cost, and that can be accessed by anyone. These behavioral barriers for civil servants influence governments’ decisions to make data available to the public. Behavioral barriers are the impediments for governments to release open data which originates from human behaviors. The literature suggests that behaviors are difficult to measure, and therefore, we focus on attitudes, which are measurable through declared perception. Attitude refers to a set of beliefs and feelings which is a common predictor of behavior. In this research, we use governmental civil servants’ behavioral intention to support open data to measure attitudes and the change in behavior intentions of civil servants as a proxy to analyze attitude change. Serious games are game-based interventions designed for other goals than (only) entertaining the players. They offer a safe and controlled environment for experimentation and experiential learning. The research objective of this thesis is to develop and test a game to influence the attitudes of civil servants towards the release of open data by governments, by enabling them to experience the positive and negative sides of open data in the game. Design science research was used for prototyping development and testing a game in a quasi-experimental set-up. Four research questions guided the study: RQ1. What are the behavioral barriers for civil servants to support the opening of governmental data? RQ2. What are the requirements to design a game to change civil servants’ attitudes towards supporting the opening of governmental data? RQ3. Which game design mechanisms enable the change of civil servants’ attitudes towards opening governmental data? RQ4. What are the effects of the open data game on civil servant’s attitudes towards supporting the opening of data? Each research question demanded the application of specific research methods. As the first step, systematic literature reviews were performed in the field of 1) open data provision behavioral barriers, 2) games for civil servants, 3) games for open data, and 4) games designed for attitude change. The first literature review was used to answer RQ1, whereas the other aimed at RQ2. For RQ1 (What are the behavioral barriers for civil servants to support the opening of governmental data?), the literature review identified a list of 38 behavioral barriers for civil servants influencing the opening of data. These behavioral barriers discussed in this thesis should be considered to change civil servants’ attitudes to support the opening of governmental data. For RQ2 (What are the requirements to design a game to change civil servants’ attitudes towards supporting the opening of governmental data?), three literature reviews were conducted to find game design requirements from previous research. They targeted at specific aspects of proven serious games: 1) for civil servants, to better understand the audience characteristics which could influence gameplay; 2) using open data content to inspire metaphors and operational representation of data release in the game; and 3) to change attitudes of players, targeting at successful use of game use towards attitude change. For civil servants, many games exist, whereas, for open data provision, no games were found. Even though many mechanisms exist in the literature, they did not prescribe an operationalization for an open data game. To evolve towards the most suitable game, we followed an iterative process to better understand how the game could be realized. Games are context-dependent, particularly to our specific case, open data governmental provision. Likewise, the iterative process enabled testing the operationalization of such requirements into game mechanisms. Four prototypes resulted from this game design process. Each designed prototype was evaluated, updating the lists of requirements and mechanisms for the final version of the game. •Prototype 1: Cards for open data debriefing showed that engaging mechanics could help connecting players to the open data challenges, but a card game resulted in lower levels of knowledge transfer about open data; •Prototype 2. Solvd, a group debate play-setting, resulted in interactive content from group interactions. However, the game was not entertaining, resulting in a loss of engagement; •Prototype 3. Job-matching simulator, a decision-making labor-market digital game, helped to map the real-life public service data production and use routines. This prototype highlighted the need to represent situations encountered by public servants in reality, including risks and ways to prevent them; and •Prototype 4. Open data office, a role-playing game aimed at engagement and learning for attitude change. Still, it lacked a more precise metaphor for routines and the office environment. Likewise, playing roles with humans was found to be important for our learning goals, in addition to adjusting the number of players and rounds. The prototypes resulted in the following main requirements on a serious game to influence civil servants support to the opening of data: •Requirement 1. Open government data content used in the game should be highlighted; •Requirement 2. The focus should be on a game experience that enables experiential learning; •Requirement 3. Civil servants’ practical knowledge should be reflected in the game; •Requirement 4. The game should be used as a safe environment for experimentation; •Requirement 5. The game setting should be realistic; •Requirement 6. Game dynamics should be organized as a role-playing game; and •Requirement 7. The number of roles, players, and rounds should be limited. Additionally, the literature findings combined with the outcomes of the iterative design cycles, pilot-testing, and debriefing, enabled the answering of RQ3 (Which game design mechanisms enable the change of civil servants’ attitudes towards opening governmental data?). The final version of the game, named WINNING DATA, operationalized the requirements into mechanisms that enabled players to change their attitudes towards open data. These mechanisms emerged from the design process, where each prototype debriefing informed the next round of iteration and new prototype. For instance, the needs for open data content and realism are represented through assets such as forms, files, and demand cards; demand cards express pre-defined routines: service requests. Demands are identified by specific card codes, which enable an automatic scoring system for the game facilitation; the service delivery, processed by rolling sets of dice, results in the creation of datasets. Depending on the dice combinations, privacy and security crises can occur, affecting the challenges of the game. The following final list of mechanisms resulted from this process: •Mechanism 1: Dataset description and labeling; •Mechanism 2: Card codes; •Mechanism 3: Pre-defined demands (not random); •Mechanism 4: Forms, Files and Demand cards; •Mechanism 5: Service delivery goal; •Mechanism 6: Upgrades; •Mechanism 7: Facilitation; •Mechanism 8: Crisis board; •Mechanism 9: Dice as processing machine; •Mechanism 10: Multi-player (with different roles); and •Mechanism 11: Time-limited rounds. Based on these requirements and mechanisms, WINNING DATA was designed as a four-player role-playing in-person game that can be played in a two-hour session. The game was evaluated for its effects on the attitudes of civil servants towards supporting the opening of governmental data. Playing the game consists of five rounds in which participants switch roles. The roles are citizen, two civil servants, and a manager. The player, playing the role of a citizen, demands services to the one playing the role of a civil servant; the player playing the role of civil servant has to work together with the colleague and boss to deliver the service back. Each service delivered results in a dataset which is discussed by the team and labeled by the boss. Labeling decisions influence the chances of having a privacy or security crisis in the coming rounds, resulting from specific dice combinations. Lastly, game play, data collection, and statistical analysis were used to answer the RQ4 (What are the effects of the open data game on civil servant’s attitudes towards supporting the opening of data?). Our main hypothesis is that the attitudes of civil servants can be changed by using a serious game. From the list of behavior barriers (RQ1), an initial list of factors influencing civil servants’ attitudes emerged. Four influencing factors were defined to influence Behavioral Intention, the dependent variable representing civil servants’ attitudes: lack of knowledge, performance expectancy, effort expectancy, and social influence. Explorative testing was conducted to determine which factors are at work and how the game affected them. The factors were hypothesized for testing game effects on civil servants’ attitudes to supporting open data. All factors were measured using a 33-item 7-point Likert scale questionnaire. The survey was used to measure the players’ attitudes before and after the game was played. Comparison enabled the assessment of the effects of change in their attitudes. In a quasi-experimental set-up, 77 civil servants played the game and filled in the pre- and post-test survey. Another 35 civil servants filled in the survey on two different occasions, without the gaming intervention, as a control group. The data was analyzed. Firstly, the internal reliability of the factors was checked, followed by explorative testing on the factors that did not load. The resulting factors were organized into a model which included Behavioral Intention as the dependent factor, measuring multiple dimensions of civil servants’ attitudes towards open data. Other seven factors were defined: Data Management Knowledge (DK), Performance Expectancy (PE), Risks (RK), Social Influence (SI), Knowledge of Data Production (DP), Data Sharing Knowledge (DS), and Data Costs (DC). The eight resulting hypotheses were tested using the 112 completed surveys: Hypothesis 1: Behavioral intention increases after playing the game; Hypothesis 2: The game results in more knowledge about ways to open data; Hypothesis 3: The game results in a better understanding of the expected benefits of opening data; Hypothesis 4: The game decreases expectations of the risks related to making data available; Hypothesis 5: The game reduces civil servants’ perceptions of open data practice difficulties, as exerted by hierarchies and legal frameworks; Hypothesis 6: The game increases civil servants’ knowledge of data production; Hypothesis 7: The game increases civil servants’ knowledge of the possibility of sharing data; and Hypothesis 8: The game increases civil servants’ perception of data provision costs. Through a Wilcoxon Signed Rank test, we assessed the main hypothesis and concluded that the game is likely to have a statistically significant effect on the dependent variable of Behavior Intention. As we did not find significant effects on behavior intention in the control group, our conclusion that civil servants who played the game are likely to have their attitudes towards open data increased by the game was strengthened. After that, the WINNING DATA’s gameplay additional seven hypotheses were tested. The game had a significant positive effect on Risks and Performance Expectancy. Though there were differences in the pre- and post-test scores of Data management knowledge, Social Influence, Knowledge of Data Production, Data Sharing Knowledge, and Data Costs, none of them were statistically significant. Our research has limitations resulting from (1) the limited number of participants and their distributions’ characteristics; (2) the absence of alternative strategies to which our results could be compared; and (3) the feasibility of more complex statistical analyses that were limited due to the available sample. Furthermore, this research (4) could not explore other diverse outcomes, such as a more complex model discussion on the factors influencing civil servants’ attitudes to support the opening of governmental data, which is needed and still to be done. Additionally, these limitations shed light on other improvements for new versions of the game. Future research is recommended to test the game with larger samples, players having a more diverse background, and coming from different countries. Using the same survey questions to different passive interventions, such as text and lectures, can also contribute to comparing the results. The long-term effects of the game were not investigated and recommended as a further research direction. Another further research direction is the digitalization of the game. Particularly in the light of the recent crisis of COVID-19, this is needed as playing the game with many persons in one room is not a good option. Likewise, advancing with the model discussions, including more open data elements, and extending the topics to other fields is also recommended by this thesis. Concluding, the game developed and tested during this project has proven its effects on changing civil servants’ attitudes towards the opening of governmental data. This thesis’s results can be used to design better interventions to make more governmental data available to the public.
Original languageEnglish
QualificationDoctor of Philosophy
Awarding Institution
  • Delft University of Technology
Supervisors/Advisors
  • Janssen, M.F.W.H.A., Supervisor
  • Meijer, S.A., Supervisor
Thesis sponsors
Award date23 Sep 2021
Print ISBNs978-903610662-7
DOIs
Publication statusPublished - 2021

Funding

This research was funded by the Brazilian Federal government through the Long-term qualification Programme, Ministry of Economy.

Keywords

  • Open data
  • Attitude
  • Attitude change
  • Serious Gaming
  • Serious Game
  • Open Government
  • OGD
  • Game Design

Fingerprint

Dive into the research topics of 'Winning Data: Designing and testing a game to change civil servants' attitudes towards open governmental data provision'. Together they form a unique fingerprint.

Cite this