Fluorescence lifetime imaging endomicroscopy based ex-vivo lung cancer prediction using multi-scale concatenated-dilation convolutional neural networks

Qiang Wang, James R. Hopgood, Marta Vallejo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Downloads (Pure)

Abstract

Deep learning technologies have been successfully applied to automatic diagnostics of ex-vivo lung cancer with fluorescence lifetime imaging endomicroscopy (FLIM). Recent advance in convolutional neural networks (CNNs) by splitting input features for multi-scale feature extraction as a feature-level aggregation, has achieved further improvement in visual recognition. However, due to the splitting, correlations among input features are no longer retained. To exploit the advantages of hierarchical multi-scale architectures, while maintaining the correlations as global information, we propose a novel architecture, namely multi-scale concatenated-dilation (MSCD) at a layer level. The MSCD performs multi-scale feature extraction on input features without the splitting. In addition, we substitute the Addition aggregation in the original hierarchical architecture with the Concatenation to retrieve more features. At the same time, we also introduce dilated convolutions to replace the linear convolutions to further enlarge the receptive field. We evaluate the performance of MSCD by integrating it into ResNet, on over 60,000 FLIM images collected from 14 patients, using a custom fiber-based FLIM system, with various user-specified configurations. Accuracy, precision, recall, and the area under the receiver operating characteristic curve are used as the metrics. We first demonstrate the superiority of our MSCD model over the backbone ResNet and other state-of-the-art CNNs in terms of higher scores with lower complexity over the metrics. Moreover, we empirically demonstrate the superiority of the Concatenation aggregation over the Addition on convolution and scale efficiency. Furthermore, we compare the MSCD with Res2Net to illustrate the advantages and disadvantages of feature-/layer-level multi-scale aggregation.
Original languageEnglish
Title of host publicationMedical Imaging 2021
Subtitle of host publicationComputer-Aided Diagnosis
Editors Maciej A. Mazurowski, Karen Drukker
PublisherSPIE
ISBN (Electronic)9781510640238
DOIs
Publication statusPublished - 15 Feb 2021
EventSPIE Medical Imaging -
Duration: 15 Feb 202119 Feb 2021
https://spie.org/conferences-and-exhibitions/medical-imaging

Publication series

NameProceedings of SPIE
Volume11597
ISSN (Print)0277-786X
ISSN (Electronic)0277-786X

Conference

ConferenceSPIE Medical Imaging
Period15/02/2119/02/21
Internet address

Keywords

  • Convolutional neural networks
  • Dilated convolutions
  • FLIM
  • Fluorescence lifetime endo/microscopic images
  • Lung cancer prediction
  • MSCD
  • Multi-scale concatenated dilation

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Atomic and Molecular Physics, and Optics
  • Radiology Nuclear Medicine and imaging

Fingerprint

Dive into the research topics of 'Fluorescence lifetime imaging endomicroscopy based ex-vivo lung cancer prediction using multi-scale concatenated-dilation convolutional neural networks'. Together they form a unique fingerprint.

Cite this