CS2: A Controllable and Simultaneous Synthesizer of Images and Annotations with Minimal Human Intervention

Xiaodan Xing, Jiahao Huang, Yang Nan, Yinzhe Wu, Chengjia Wang, Zhifan Gao, Simon Walsh, Guang Yang*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The destitution of image data and corresponding expert annotations limit the training capacities of AI diagnostic models and potentially inhibit their performance. To address such a problem of data and label scarcity, generative models have been developed to augment the training datasets. Previously proposed generative models usually require manually adjusted annotations (e.g., segmentation masks) or need pre-labeling. However, studies have found that these pre-labeling based methods can induce hallucinating artifacts, which might mislead the downstream clinical tasks, while manual adjustment could be onerous and subjective. To avoid manual adjustment and pre-labeling, we propose a novel controllable and simultaneous synthesizer (dubbed CS2) in this study to generate both realistic images and corresponding annotations at the same time. Our CS2 model is trained and validated using high resolution CT (HRCT) data collected from COVID-19 patients to realize an efficient infections segmentation with minimal human intervention. Our contributions include 1) a conditional image synthesis network that receives both style information from reference CT images and structural information from unsupervised segmentation masks, and 2) a corresponding segmentation mask synthesis network to automatically segment these synthesized images simultaneously. Our experimental studies on HRCT scans collected from COVID-19 patients demonstrate that our CS2 model can lead to realistic synthesized datasets and promising segmentation results of COVID infections compared to the state-of-the-art nnUNet trained and fine-tuned in a fully supervised manner.
Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention. MICCAI 2022
EditorsLinwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li
PublisherSpringer
Pages3-12
Number of pages10
ISBN (Electronic)9783031164521
ISBN (Print)9783031164514
DOIs
Publication statusPublished - 16 Sep 2022
Event25th International Conference on Medical Image Computing and Computer-Assisted Intervention 2022 - Singapore, Singapore
Duration: 18 Sep 202222 Sep 2022

Publication series

NameLecture Notes in Computer Science
Volume13438
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th International Conference on Medical Image Computing and Computer-Assisted Intervention 2022
Abbreviated titleMICCAI 2022
Country/TerritorySingapore
CitySingapore
Period18/09/2222/09/22

Keywords

  • Data augmentation
  • Generative model
  • Semi-supervised segmentation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'CS2: A Controllable and Simultaneous Synthesizer of Images and Annotations with Minimal Human Intervention'. Together they form a unique fingerprint.

Cite this