We propose a novel view path-planning method based on an online Multi-View Stereo (MVS) system. This method aims to incrementally construct the target three-dimensional (3D) model in real time. View paths are continually planned based on online feedback from the partially constructed model. The obtained paths fully cover low-quality surfaces while maximizing the reconstruction performance of MVS. Experimental results demonstrate that the proposed method can construct high quality 3D models with one exploration trial, without any re-scanning trial as in the explore-then-exploit method.
Multi-view stereo (MVS) is a crucial task for precise 3D reconstruction. Most recent studies tried to improve the performance of matching cost volume in MVS by designing aggregated 3D cost volumes and their regularization. we propose to learn a robust feature extraction network to enhance the performance of matching costs without heavy computation in the other steps. We built a dynamic scale feature extraction network, namely, CDSFNet. It is composed of multiple novel convolution layers, each of which can select a proper patch scale for each pixel guided by the normal curvature of the image surface. As a result, CDFSNet can estimate the optimal patch scales to learn discriminative features for accurate matching computation between reference and source images. By combining the robust extracted features with an appropriate cost formulation strategy, our resulting MVS architecture can estimate depth maps more
precisely.
Related publications
1. S Song, KT Giang, D Kim, S Jo, Prior depth-based multi-view stereo network for online 3D model reconstruction, Pattern Recognition, 136, April 2023 [LINK] [PDF]
2. KT Giang, S Song, S Jo, Curvature-Guided Dynamic Scale Networks for Multi-view Stereo, 2022 International Conference on Learning Representations (ICLR) [PDF] [VIDEO]
3. S Song, D Kim, S Choi, View Path Planning via Online Multi-View Stereo for 3D Modeling of Large-Scale Structures, IEEE Transactions on Robotics, 38(1): 372-390, 2022 [LINK] [PDF] [VIDEO]
4. KT Giang, S Song, D Kim, S Choi, Sequential Depth Completion with Confidence Estimation for 3D Model Reconstruction, IEEE RA-L, 6(2) : 327- 334, 2021 [LINK] [PDF] [VIDEO]
5. S Song, D Kim, S Jo, Active 3D modeling via online multi-view stereo, 2020 IEEE International Conference on Robotics and Automation (ICRA) [LINK] [PDF]
