Nonrigid image registration using multi-scale 3D convolutional neural networks

Hessam Sokooti*, Bob D. De Vos, Floris Berendsen, Boudewijn Lelieveldt, Ivana Išgum, Marius Staring

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

154 Citations (Scopus)


In this paper we propose a method to solve nonrigid image registration through a learning approach, instead of via iterative optimization of a predefined dissimilarity metric. We design a Convolutional Neural Network (CNN) architecture that, in contrast to all other work, directly estimates the displacement vector field (DVF) from a pair of input images. The proposed RegNet is trained using a large set of artificially generated DVFs, does not explicitly define a dissimilarity metric, and integrates image content at multiple scales to equip the network with contextual information. At testing time nonrigid registration is performed in a single shot, in contrast to current iterative methods. We tested RegNet on 3D chest CT follow-up data. The results show that the accuracy of RegNet is on par with a conventional B-spline registration, for anatomy within the capture range. Training RegNet with artificially generated DVFs is therefore a promising approach for obtaining good results on real clinical data, thereby greatly simplifying the training problem. Deformable image registration can therefore be successfully casted as a learning problem.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention - MICCAI 2017
Subtitle of host publication20th International Conference, Proceedings
EditorsM. Descoteaux, L. Maier-Hein, A. Franz, P. Jannin, D. Louis Collins, S. Duchesne
Place of PublicationCham
Number of pages8
EditionPart 1
ISBN (Electronic)978-3-319-66182-7
ISBN (Print)978-3-319-66181-0
Publication statusPublished - 2017
EventMedical Image Computing and Computer-Assisted Intervention, MICCAI 2017: 20th International Conference - Quebec City, Canada
Duration: 11 Sep 201713 Sep 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceMedical Image Computing and Computer-Assisted Intervention, MICCAI 2017
CityQuebec City


  • Chest CT
  • Convolutional neural networks
  • Image registration
  • Multi-scale analysis


Dive into the research topics of 'Nonrigid image registration using multi-scale 3D convolutional neural networks'. Together they form a unique fingerprint.

Cite this