Bayesian Neural Nets (BNN) are increasingly used for robust organ auto-contouring. Uncertainty heatmaps extracted from BNNs have been shown to correspond to inaccurate regions. To help speed up the mandatory quality assessment (QA) of contours in radiotherapy, these heatmaps could be used as stimuli to direct visual attention of clinicians to potential inaccuracies. In practice, this is non-trivial to achieve since many accurate regions also exhibit uncertainty. To influence the output uncertainty of a BNN, we propose a modified accuracy-versus-uncertainty (AvU) metric as an additional objective during model training that penalizes both accurate regions exhibiting uncertainty as well as inaccurate regions exhibiting certainty. For evaluation, we use an uncertainty-ROC curve that can help differentiate between Bayesian models by comparing the probability of uncertainty in inaccurate versus accurate regions. We train and evaluate a FlipOut BNN model on the MICCAI2015 Head and Neck Segmentation challenge dataset and on the DeepMind-TCIA dataset, and observed an increase in the AUC of uncertainty-ROC curves by 5.6% and 5.9%, respectively, when using the AvU objective. The AvU objective primarily reduced false positives regions (uncertain and accurate), drawing less visual attention to these regions, thereby potentially improving the speed of error detection.
|Title of host publication||Uncertainty for Safe Utilization of Machine Learning in Medical Imaging - 4th International Workshop, UNSURE 2022, Held in Conjunction with MICCAI 2022, Proceedings|
|Editors||Carole H. Sudre, Carole H. Sudre, Christian F. Baumgartner, Adrian Dalca, Adrian Dalca, William M. Wells III, Chen Qin, Ryutaro Tanno, Koen Van Leemput, Koen Van Leemput, William M. Wells III|
|Number of pages||10|
|Publication status||Published - 2022|
|Event||4th Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2022, held in conjunction with 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022 - Singapore, Singapore|
Duration: 18 Sep 2022 → 18 Sep 2022
|Name||Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)|
|Conference||4th Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2022, held in conjunction with 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022|
|Period||18/09/22 → 18/09/22|
Bibliographical noteGreen Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
- AvU loss
- Bayesian uncertainty
- Deep learning
- Quality assessment