Hierarchical Prediction of Registration Misalignment using a Convolutional LSTM: Application to Chest CT Scans

Hessam Sokooti, Sahar Yousefi, Mohamed S. Elmahdy, Boudewijn P.F. Lelieveldt, Marius Staring

Research output: Contribution to journalArticleScientificpeer-review

2 Citations (Scopus)
19 Downloads (Pure)


In this paper we propose a supervised method to predict registration misalignment using convolutional neural networks (CNNs). This task is casted to a classification problem with multiple classes of misalignment: 'correct' 0-3 mm, 'poor' 3-6 mm and 'wrong' over 6 mm. Rather than a direct prediction, we propose a hierarchical approach, where the prediction is gradually refined from coarse to fine. Our solution is based on a convolutional Long Short-Term Memory (LSTM), using hierarchical misalignment predictions on three resolutions of the image pair, leveraging the intrinsic strengths of an LSTM for this problem. The convolutional LSTM is trained on a set of artificially generated image pairs obtained from artificial displacement vector fields (DVFs). Results on chest CT scans show that incorporating multi-resolution information, and the hierarchical use via an LSTM for this, leads to overall better F1 scores, with fewer misclassifications in a well-tuned registration setup. The final system yields an accuracy of 87.1%, and an average F1 score of 66.4% aggregated in two independent chest CT scan studies.

Original languageEnglish
Article number9408621
Pages (from-to)62008-62020
Number of pages13
JournalIEEE Access
Publication statusPublished - 2021


  • Biomedical imaging
  • Computed tomography
  • convolutional neural networks
  • Decoding
  • Feature extraction
  • hierarchical classification
  • Image registration
  • image registration
  • Image resolution
  • registration misalignment
  • Task analysis


Dive into the research topics of 'Hierarchical Prediction of Registration Misalignment using a Convolutional LSTM: Application to Chest CT Scans'. Together they form a unique fingerprint.

Cite this