Prediction Models for Individuals' Control Skill Development and Retention using XGBoost and SHAP

B.A.A. van Leeuwen, Armon Toubman, Jelke van der Pal, D.M. Pool

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

66 Downloads (Pure)

Abstract

Current capabilities for predicting skill retention, i.e., the extent to which human operators retain learned skills over time, at an individual level are limited due to a requirement for large data sets and methods that can extract relevant patterns in highly dimensional data. This paper investigates the application of Extreme Gradient Boosting (XGBoost) decision tree models for predicting a high-resolution individual skill retention curve. For this, a large skill-based tracking experiment dataset is used to extract different feature classes and train an XGBoost predictive model. To identify the robust predictors, the effects of the different features on the model's output are analyzed using SHapley Additive exPlanations (SHAP). Furthermore, the proposed XGBoost model is trained using both the experiment dataset and a matched synthetic dataset, with both approaches evaluated on the experiment data. Overall, the available experiment dataset was found to include too few retention measurements, and too significant between-group differences, to extract a reliable prediction model. On the synthetic dataset, the XGBoost model was found to accurately capture individuals' skill retention curves, where the features that contributed most (21%) to the prediction model's accuracy were found to be the considered learning curve parameters. Overall, this paper shows that experiment data of skill-based tracking tasks can be used to predict skill decay curves using XGBoost, but that more research and data are needed to achieve sufficient accuracy and reliability at an individual level for practical applications.
Original languageEnglish
Title of host publicationAIAA SciTech Forum 2023
Number of pages25
ISBN (Electronic)978-1-62410-699-6
DOIs
Publication statusPublished - 2023
EventAIAA SCITECH 2023 Forum - National Harbor, MD & Online, Washington, United States
Duration: 23 Jan 202327 Jan 2023
https://arc-aiaa-org.tudelft.idm.oclc.org/doi/book/10.2514/MSCITECH23

Conference

ConferenceAIAA SCITECH 2023 Forum
Country/TerritoryUnited States
CityWashington
Period23/01/2327/01/23
Internet address

Fingerprint

Dive into the research topics of 'Prediction Models for Individuals' Control Skill Development and Retention using XGBoost and SHAP'. Together they form a unique fingerprint.

Cite this