Multi-view Contour-constrained Transformer Network for Thin-cap Fibroatheroma Identification

Sijie Liu, Jingmin Xin*, Jiayi Wu, Yangyang Deng, Ruisheng Su, Wiro J. Niessen, Nanning Zheng, Theo van Walsum

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

38 Downloads (Pure)

Abstract

Identification and detection of thin-cap fibroatheroma (TCFA) from intravascular optical coherence tomography (IVOCT) images is critical for treatment of coronary heart diseases. Recently, deep learning methods have shown promising successes in TCFA identification. However, most methods usually do not effectively utilize multi-view information or incorporate prior domain knowledge. In this paper, we propose a multi-view contour-constrained transformer network (MVCTN) for TCFA identification in IVOCT images. Inspired by the diagnosis process of cardiologists, we use contour constrained self-attention modules (CCSM) to emphasize features corresponding to salient regions (i.e., vessel walls) in an unsupervised manner and enhance the visual interpretability based on class activation mapping (CAM). Moreover, we exploit transformer modules (TM) to build global-range relations between two views (i.e., polar and Cartesian views) to effectively fuse features at multiple feature scales. Experimental results on a semi-public dataset and an in-house dataset demonstrate that the proposed MVCTN outperforms other single-view and multi-view methods. Lastly, the proposed MVCTN can also provide meaningful visualization for cardiologists via CAM.

Original languageEnglish
Pages (from-to)224-234
JournalNeurocomputing
Volume523
DOIs
Publication statusPublished - 2023

Keywords

  • IVOCT
  • Multi-view learning
  • Plaque identification
  • TCFA
  • Transformer

Fingerprint

Dive into the research topics of 'Multi-view Contour-constrained Transformer Network for Thin-cap Fibroatheroma Identification'. Together they form a unique fingerprint.

Cite this