Quantity-Aware Coarse-to-Fine Correspondence for Image-to-Point Cloud Registration

Gongxin Yao, Yixin Xuan, Yiwei Chen, Yu Pan

Research output: Contribution to journalArticlepeer-review

21 Downloads (Pure)

Abstract

Image-to-point cloud registration aligns the data acquired by heterogeneous sensors like camera and LiDAR. To capture cross-modal correspondences for estimating the relative pose, matching points with pixels presents great ambiguity due to the disparate visual attributes. This paper focuses on high-level semantic matching between point sets and pixel patches to alleviate the ambiguity, while accurately refining the coarse results to the point and pixel levels. However, it is impossible to generate perfectly aligned point sets and pixel patches when self-organizing the elements without camera pose priors, thus introducing numerous outliers during fine-level matching. Therefore, we propose a learning framework to bind the quantity of points with the degree of semantic correlation between point sets and pixel patches. The learned quantity-aware correspondences play two critical roles: indexing pixel patches for each point set based on their degree of correlation, and estimating the number of outliers from point sets to pixel patches. Specifically, we learn local semantic proxies to estimate the cost of the optimal transport problem from point sets to pixel patches, where the points are viewed as allocatable entities and patches as receptacles. A novel supervision strategy is then proposed to quantify the degree of correlation as continuous values. Afterwards, point-to-pixel correspondences are refined from a smaller search space by a resample-then-filter scheme, with a confidence sorting strategy to adaptively remove outliers based on the quantity-aware priors. Extensive experiments on the KITTI Odometry and NuScenes benchmarks demonstrate the superiority of our method over the state-of-the-art methods.
Original languageEnglish
Pages (from-to)33826-33837
Number of pages12
JournalIEEE Sensors Journal
Volume24
Issue number20
Early online date11 Sept 2024
DOIs
Publication statusPublished - 15 Oct 2024

Keywords

  • Heterogeneous sensors
  • data registration
  • feature matching
  • pose estimation

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Quantity-Aware Coarse-to-Fine Correspondence for Image-to-Point Cloud Registration'. Together they form a unique fingerprint.

Cite this