Contextual loss functions for optimization of convolutional neural networks generating pseudo CTs from MRI

M. van Stralen, Y. Zhou, P.J. Wozny, P.R. Seevinck, M. Loog

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

4 Citations (Scopus)


Predicting pseudo CT images from MRI data has received increasing attention for use in MRI-only radiation therapy planning and PET-MRI attenuation correction, eliminating the need for harmful CT scanning. Current approaches focus on voxelwise mean absolute error (MAE) and peak signal-to-noise-ratio (PSNR) for optimization and evaluation. Contextual losses such as structural similarity (SSIM) are known to promote perceptual image quality. We investigate the use of these contextual losses for optimization. Patch-based 3D fully convolutional neural networks (FCN) were optimized for prediction of pseudo CT images from 3D gradient echo pelvic MRI data and compared to ground truth CT data of 26 patients. CT data was non-rigidly registered to MRI for training and evaluation. We compared voxelwise L1 and L2 loss functions, with contextual multi-scale L1 and L2 (MSL1 and MSL2), and SSIM. Performance was evaluated using MAE, PSNR, SSIM and the overlap of segmented cortical bone in the reconstructions, by the dice similarity metric. Evaluation was carried out in cross-validation. All optimizations successfully converged well with PSNR between 25 and 30 HU, except for one of the folds of SSIM optimizations. MSL1 and MSL2 are at least on par with their single-scale counterparts. MSL1 overcomes some of the instabilities of the L1 optimized prediction models. MSL2 optimization is stable, and on average, outperforms all the other losses, although quantitative evaluations based on MAE, PSNR and SSIM only show minor differences. Direct optimization using SSIM visually excelled in terms subjective perceptual image quality at the expense of a voxelwise quantitative performance drop. Contextual loss functions can improve prediction performance of FCNs without change of the network architecture. The suggested subjective superiority of contextual losses in reconstructing local structures merits further investigations.

Original languageEnglish
Title of host publicationMedical Imaging 2018
Subtitle of host publicationImage Processing
EditorsE.D. Angelini, B.A. Landman
Place of PublicationBellingham
Pages105741N-1 - 105741N-6
Number of pages6
ISBN (Electronic)9781510616387
ISBN (Print)9781510616370
Publication statusPublished - 2018
EventSPIE Medical Imaging 2018: Image Processing - Houston, United States
Duration: 10 Feb 201815 Feb 2018

Publication series

NameProceedings of Spie
ISSN (Print)1605-7422
ISSN (Electronic)2410-9045


ConferenceSPIE Medical Imaging 2018
CountryUnited States
Internet address


  • contextual loss functions
  • Convolutional neural networks
  • magnetic resonance imaging
  • pseudo CT
  • structural similarity


Dive into the research topics of 'Contextual loss functions for optimization of convolutional neural networks generating pseudo CTs from MRI'. Together they form a unique fingerprint.

Cite this