TY - JOUR
T1 - DiCyc
T2 - GAN-based deformation invariant cross-domain information fusion for medical image synthesis
AU - Wang, Chengjia
AU - Yang, Guang
AU - Papanastasiou, Giorgos
AU - Tsaftaris, Sotirios A.
AU - Newby, David E.
AU - Gray, Calum
AU - Macnaught, Gillian
AU - MacGillivray, Tom J.
N1 - Funding Information:
This work is funded by British Heart Foundation, UK (no. RG/16/10/32375 ). D.E. Newby is supported by the British Heart Foundation ( CH/09/002 , RG/16/10/32375 , RE/18/5/34216 ) and is the recipient of a Wellcome Trust Senior Investigator Award ( WT103782AIA ). G. Yang is supported by IIAT Hangzhou, the British Heart Foundation (Grant Number: PG/16/78/32402 ), the European Research Council Innovative Medicines Initiative on Development of Therapeutics and Diagnostics Combatting Coronavirus Infections Award ( H2020-JTI-IMI2 101005122 ), and the AI for Health Imaging Award ( H2020-SC1-FA-DTS-2019-1 952172 ). S.A. Tsaftaris and G. Papanastasiou acknowledge support from the EPSRC, UK Grant ( EP/P022928/1 ). S.A. Tsaftaris acknowledges the support of the Royal Academy of Engineering, UK and the Research Chairs and Senior Research Fellowships scheme .
Publisher Copyright:
© 2020 The Authors
PY - 2021/3
Y1 - 2021/3
N2 - Cycle-consistent generative adversarial network (CycleGAN) has been widely used for cross-domain medical image synthesis tasks particularly due to its ability to deal with unpaired data. However, most CycleGAN-based synthesis methods cannot achieve good alignment between the synthesized images and data from the source domain, even with additional image alignment losses. This is because the CycleGAN generator network can encode the relative deformations and noises associated to different domains. This can be detrimental for the downstream applications that rely on the synthesized images, such as generating pseudo-CT for PET-MR attenuation correction. In this paper, we present a deformation invariant cycle-consistency model that can filter out these domain-specific deformation. The deformation is globally parameterized by thin-plate-spline (TPS), and locally learned by modified deformable convolutional layers. Robustness to domain-specific deformations has been evaluated through experiments on multi-sequence brain MR data and multi-modality abdominal CT and MR data. Experiment results demonstrated that our method can achieve better alignment between the source and target data while maintaining superior image quality of signal compared to several state-of-the-art CycleGAN-based methods.
AB - Cycle-consistent generative adversarial network (CycleGAN) has been widely used for cross-domain medical image synthesis tasks particularly due to its ability to deal with unpaired data. However, most CycleGAN-based synthesis methods cannot achieve good alignment between the synthesized images and data from the source domain, even with additional image alignment losses. This is because the CycleGAN generator network can encode the relative deformations and noises associated to different domains. This can be detrimental for the downstream applications that rely on the synthesized images, such as generating pseudo-CT for PET-MR attenuation correction. In this paper, we present a deformation invariant cycle-consistency model that can filter out these domain-specific deformation. The deformation is globally parameterized by thin-plate-spline (TPS), and locally learned by modified deformable convolutional layers. Robustness to domain-specific deformations has been evaluated through experiments on multi-sequence brain MR data and multi-modality abdominal CT and MR data. Experiment results demonstrated that our method can achieve better alignment between the source and target data while maintaining superior image quality of signal compared to several state-of-the-art CycleGAN-based methods.
KW - GAN
KW - Image synthesis
KW - Information fusion
UR - http://www.scopus.com/inward/record.url?scp=85094323019&partnerID=8YFLogxK
U2 - 10.1016/j.inffus.2020.10.015
DO - 10.1016/j.inffus.2020.10.015
M3 - Article
C2 - 33658909
AN - SCOPUS:85094323019
SN - 1566-2535
VL - 67
SP - 147
EP - 160
JO - Information Fusion
JF - Information Fusion
ER -