Multimodal Cardiac Segmentation Using Disentangled Representation Learning

Agisilaos Chartsias; Giorgos Papanastasiou; Chengjia Wang; Colin Stirrat; Scott Semple; David Newby; Rohan Dharmakumar; Sotirios A. Tsaftaris

doi:10.1007/978-3-030-39074-7_14

Multimodal Cardiac Segmentation Using Disentangled Representation Learning

Agisilaos Chartsias^*
, Giorgos Papanastasiou
, Chengjia Wang
, Colin Stirrat
, Scott Semple
, David Newby
, Rohan Dharmakumar
, Sotirios A. Tsaftaris

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Citations (Scopus)

Abstract

Magnetic Resonance (MR) protocols use several sequences to evaluate pathology and organ status. Yet, despite recent advances, the analysis of each sequence’s images (modality hereafter) is treated in isolation. We propose a method suitable for multimodal and multi-input learning and analysis, that disentangles anatomical and imaging factors, and combines anatomical content across the modalities to extract more accurate segmentation masks. Mis-registrations between the inputs are handled with a Spatial Transformer Network, which non-linearly aligns the (now intensity-invariant) anatomical factors. We demonstrate applications in Late Gadolinium Enhanced (LGE) and cine MRI segmentation. We show that multi-input outperforms single-input models, and that we can train a (semi-supervised) model with few (or no) annotations for one of the modalities. Code is available at https://github.com/agis85/multimodal_segmentation.

Original language	English
Title of host publication	Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges
Subtitle of host publication	10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers
Editors	Mihaela Pop, Maxime Sermesant, Oscar Camara, Xiahai Zhuang, Shuo Li, Alistair Young, Tommaso Mansi, Avan Suinesiaputra
Publisher	Springer
Pages	128-137
Number of pages	10
Volume	12009
ISBN (Electronic)	978-3-030-39074-7
ISBN (Print)	978-3-030-39073-0
DOIs	https://doi.org/10.1007/978-3-030-39074-7_14
Publication status	Published - 23 Jan 2020
Event	10th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2019 - Shenzhen, China Duration: 13 Oct 2019 → 13 Oct 2019

Publication series

Name	Lecture Notes in Computer Science
Volume	12009
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	10th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2019
Country/Territory	China
City	Shenzhen
Period	13/10/19 → 13/10/19

Keywords

Cardiac MR
Disentanglement
Multimodal segmentation
Representation learning

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-39074-7_14

Cite this

Chartsias, A., Papanastasiou, G., Wang, C., Stirrat, C., Semple, S., Newby, D., Dharmakumar, R., & Tsaftaris, S. A. (2020). Multimodal Cardiac Segmentation Using Disentangled Representation Learning. In M. Pop, M. Sermesant, O. Camara, X. Zhuang, S. Li, A. Young, T. Mansi, & A. Suinesiaputra (Eds.), Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges : 10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers (Vol. 12009, pp. 128-137). (Lecture Notes in Computer Science; Vol. 12009). Springer. https://doi.org/10.1007/978-3-030-39074-7_14

Chartsias, Agisilaos ; Papanastasiou, Giorgos ; Wang, Chengjia et al. / Multimodal Cardiac Segmentation Using Disentangled Representation Learning. Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges : 10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers. editor / Mihaela Pop ; Maxime Sermesant ; Oscar Camara ; Xiahai Zhuang ; Shuo Li ; Alistair Young ; Tommaso Mansi ; Avan Suinesiaputra. Vol. 12009 Springer, 2020. pp. 128-137 (Lecture Notes in Computer Science).

@inproceedings{7fd67063315e4108b06d01c32a7425bd,

title = "Multimodal Cardiac Segmentation Using Disentangled Representation Learning",

abstract = "Magnetic Resonance (MR) protocols use several sequences to evaluate pathology and organ status. Yet, despite recent advances, the analysis of each sequence{\textquoteright}s images (modality hereafter) is treated in isolation. We propose a method suitable for multimodal and multi-input learning and analysis, that disentangles anatomical and imaging factors, and combines anatomical content across the modalities to extract more accurate segmentation masks. Mis-registrations between the inputs are handled with a Spatial Transformer Network, which non-linearly aligns the (now intensity-invariant) anatomical factors. We demonstrate applications in Late Gadolinium Enhanced (LGE) and cine MRI segmentation. We show that multi-input outperforms single-input models, and that we can train a (semi-supervised) model with few (or no) annotations for one of the modalities. Code is available at https://github.com/agis85/multimodal\_segmentation.",

keywords = "Cardiac MR, Disentanglement, Multimodal segmentation, Representation learning",

author = "Agisilaos Chartsias and Giorgos Papanastasiou and Chengjia Wang and Colin Stirrat and Scott Semple and David Newby and Rohan Dharmakumar and Tsaftaris, \{Sotirios A.\}",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2020.; 10th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2019 ; Conference date: 13-10-2019 Through 13-10-2019",

year = "2020",

month = jan,

day = "23",

doi = "10.1007/978-3-030-39074-7\_14",

language = "English",

isbn = "978-3-030-39073-0",

volume = "12009",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "128--137",

editor = "Mihaela Pop and Maxime Sermesant and Oscar Camara and Xiahai Zhuang and Shuo Li and Alistair Young and Tommaso Mansi and Avan Suinesiaputra",

booktitle = "Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges",

address = "Germany",

}

Chartsias, A, Papanastasiou, G, Wang, C, Stirrat, C, Semple, S, Newby, D, Dharmakumar, R & Tsaftaris, SA 2020, Multimodal Cardiac Segmentation Using Disentangled Representation Learning. in M Pop, M Sermesant, O Camara, X Zhuang, S Li, A Young, T Mansi & A Suinesiaputra (eds), Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges : 10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers. vol. 12009, Lecture Notes in Computer Science, vol. 12009, Springer, pp. 128-137, 10th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2019, Shenzhen, China, 13/10/19. https://doi.org/10.1007/978-3-030-39074-7_14

Multimodal Cardiac Segmentation Using Disentangled Representation Learning. / Chartsias, Agisilaos; Papanastasiou, Giorgos; Wang, Chengjia et al.
Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges : 10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers. ed. / Mihaela Pop; Maxime Sermesant; Oscar Camara; Xiahai Zhuang; Shuo Li; Alistair Young; Tommaso Mansi; Avan Suinesiaputra. Vol. 12009 Springer, 2020. p. 128-137 (Lecture Notes in Computer Science; Vol. 12009).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Multimodal Cardiac Segmentation Using Disentangled Representation Learning

AU - Chartsias, Agisilaos

AU - Papanastasiou, Giorgos

AU - Wang, Chengjia

AU - Stirrat, Colin

AU - Semple, Scott

AU - Newby, David

AU - Dharmakumar, Rohan

AU - Tsaftaris, Sotirios A.

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2020.

PY - 2020/1/23

Y1 - 2020/1/23

N2 - Magnetic Resonance (MR) protocols use several sequences to evaluate pathology and organ status. Yet, despite recent advances, the analysis of each sequence’s images (modality hereafter) is treated in isolation. We propose a method suitable for multimodal and multi-input learning and analysis, that disentangles anatomical and imaging factors, and combines anatomical content across the modalities to extract more accurate segmentation masks. Mis-registrations between the inputs are handled with a Spatial Transformer Network, which non-linearly aligns the (now intensity-invariant) anatomical factors. We demonstrate applications in Late Gadolinium Enhanced (LGE) and cine MRI segmentation. We show that multi-input outperforms single-input models, and that we can train a (semi-supervised) model with few (or no) annotations for one of the modalities. Code is available at https://github.com/agis85/multimodal_segmentation.

AB - Magnetic Resonance (MR) protocols use several sequences to evaluate pathology and organ status. Yet, despite recent advances, the analysis of each sequence’s images (modality hereafter) is treated in isolation. We propose a method suitable for multimodal and multi-input learning and analysis, that disentangles anatomical and imaging factors, and combines anatomical content across the modalities to extract more accurate segmentation masks. Mis-registrations between the inputs are handled with a Spatial Transformer Network, which non-linearly aligns the (now intensity-invariant) anatomical factors. We demonstrate applications in Late Gadolinium Enhanced (LGE) and cine MRI segmentation. We show that multi-input outperforms single-input models, and that we can train a (semi-supervised) model with few (or no) annotations for one of the modalities. Code is available at https://github.com/agis85/multimodal_segmentation.

KW - Cardiac MR

KW - Disentanglement

KW - Multimodal segmentation

KW - Representation learning

UR - https://www.scopus.com/pages/publications/85081906596

U2 - 10.1007/978-3-030-39074-7_14

DO - 10.1007/978-3-030-39074-7_14

M3 - Conference contribution

AN - SCOPUS:85081906596

SN - 978-3-030-39073-0

VL - 12009

T3 - Lecture Notes in Computer Science

SP - 128

EP - 137

BT - Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges

A2 - Pop, Mihaela

A2 - Sermesant, Maxime

A2 - Camara, Oscar

A2 - Zhuang, Xiahai

A2 - Li, Shuo

A2 - Young, Alistair

A2 - Mansi, Tommaso

A2 - Suinesiaputra, Avan

PB - Springer

T2 - 10th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2019

Y2 - 13 October 2019 through 13 October 2019

ER -

Chartsias A, Papanastasiou G, Wang C, Stirrat C, Semple S, Newby D et al. Multimodal Cardiac Segmentation Using Disentangled Representation Learning. In Pop M, Sermesant M, Camara O, Zhuang X, Li S, Young A, Mansi T, Suinesiaputra A, editors, Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges : 10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers. Vol. 12009. Springer. 2020. p. 128-137. (Lecture Notes in Computer Science). doi: 10.1007/978-3-030-39074-7_14

Multimodal Cardiac Segmentation Using Disentangled Representation Learning

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this