This study addresses the challenge of performing visual localization in demanding conditions such as nighttime scenarios, adverse weather, and seasonal changes. Many prior studies tend to overlook unobserved keypoints during the matching process. Therefore, dense keypoint matches are not fully exploited, leading to a notable reduction in accuracy, particularly in noisy scenes. To tackle this issue, we investigate a novel localization method that extracts reliable semi-dense 2D-3D matching points based on dense keypoint matches. This approach involves
regressing semi-dense 2D keypoints into 3D scene coordinates using a point inference network. The network utilizes both geometric and visual cues to effectively infer 3D coordinates for unobserved keypoints from the observed ones. The abundance of matching information significantly enhances the accuracy of camera pose estimation, even in scenarios involving noisy or sparse 3D models.
Related publications
1. K T Giang, S Song, S Jo, Learning to Produce Semi-dense Correspondences for Visual Localization, Conference on Computer Vision and Pattern Recognition 2024 (CVPR 2024), (within top 3.3% of accepted papers) [LINK] [PDF] [VIDEO] [CODE]
