Robust super-resolution depth imaging via a multi-feature fusion deep network

Alice Ruget, Stephen McLaughlin, Robert K. Henderson, Istvan Gyongy, Abderrahim Halimi, Jonathan Leach

Research output: Contribution to journalArticlepeer-review

27 Citations (Scopus)
69 Downloads (Pure)


The number of applications that use depth imaging is increasing rapidly, e.g. selfdriving autonomous vehicles and auto-focus assist on smartphone cameras. Light detection and ranging (LIDAR) via single-photon sensitive detector (SPAD) arrays is an emerging technology that enables the acquisition of depth images at high frame rates. However, the spatial resolution of this technology is typically low in comparison to the intensity images recorded by conventional cameras. To increase the native resolution of depth images from a SPAD camera, we develop a deep network built to take advantage of the multiple features that can be extracted from a camera’s histogram data. The network is designed for a SPAD camera operating in a dual-mode such that it captures alternate low resolution depth and high resolution intensity images at high frame rates, thus the system does not require any additional sensor to provide intensity images. The network then uses the intensity images and multiple features extracted from down-sampled histograms to guide the up-sampling of the depth. Our network provides significant image resolution enhancement and image denoising across a wide range of signal-to-noise ratios and photon levels. Additionally, we show that the network can be applied to other data types of SPAD data, demonstrating the generality of the algorithm.
Original languageEnglish
Pages (from-to)11917-11937
Number of pages21
JournalOptics Express
Issue number8
Early online date1 Apr 2021
Publication statusPublished - 12 Apr 2021


  • eess.IV
  • cs.CV

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics


Dive into the research topics of 'Robust super-resolution depth imaging via a multi-feature fusion deep network'. Together they form a unique fingerprint.

Cite this