Monocular Depth Estimation for Equirectangular Videos

Helmi Fraser, Sen Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Depth estimation from panoramic imagery has received minimal attention in contrast to standard perspective imagery, which constitutes the majority of the literature on the key research topic. The vast - and frequently complete - field of view provided by such panoramic photographs makes them appealing for a variety of applications, including robots, autonomous vehicles, and virtual reality. Consumer-level camera systems capable of capturing such images are likewise growing more affordable, and may be desirable complements to autonomous systems' sensor packages. They do, however, introduce significant distortions and violate some assumptions regarding perspective view images. Additionally, many state-of-the-art algorithms are not designed for its projection model, and their depth estimation performance tends to degrade when being applied to panoramic imagery. This paper presents a novel technique for adapting view synthesis-based depth estimation models to omnidirectional vision. Specifically, we: 1) integrate a 'virtual' spherical camera model into the training pipeline, facilitating the model training, 2) exploit spherical convolutional layers to perform convolution operations on equirectangular images, handling the severe distortion, and 3) propose an optical flow-based masking scheme to mitigate the effect of unwanted pixels during training. Our qualitative and quantitative results demonstrate that these simple yet efficient designs result in significantly improved depth estimations when compared to previous approaches.

Original languageEnglish
Title of host publication2022 IEEE/RSJ International Conference on Intelligent Robots and Systems
Number of pages7
ISBN (Electronic)9781665479271
Publication statusPublished - 26 Dec 2022
Event2022 IEEE/RSJ International Conference on Intelligent Robots and Systems - Kyoto, Japan
Duration: 23 Oct 202227 Oct 2022


Conference2022 IEEE/RSJ International Conference on Intelligent Robots and Systems
Abbreviated titleIROS 2022

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications


Dive into the research topics of 'Monocular Depth Estimation for Equirectangular Videos'. Together they form a unique fingerprint.

Cite this