Superpixel Guided Network for Three-Dimensional Stereo Matching

Superpixel provides local pixel coherence and respects object boundary, which is beneficial to stereo matching. Recently, superpixel cues are introduced into deep stereo networks. These methods develop a superpixel-based sampling scheme to downsample input color images and upsample output disparity maps. However, in this way, the image details are inevitably lost in the downsampling and the upsampling process introduces errors in the final disparity as well. Besides, this mechanism further limits the possibility of utilizing larger and multi-scale superpixels, which are important to alleviate the matching ambiguity. To address these problems, a superpixel-guided stereo matching method (LSG-Stereo) is proposed, which explicitly exploits the feature and disparity consistency within multi-scale superpixels to improve disparity estimation. To effectively incorporate superpixel cues into a stereo matching network, two novel modules are designed, including Superpixel Attention Spatial Pyramid Pooling (SA-SPP) and Superpixel-Guided Refinement (SGR). The SA-SPP module takes advantage of the content-aware superpixel pooling to construct an adaptive spatial pooling pyramid for better feature extraction. The SGR module explicitly utilizes the disparity consistency over multi-scale superpixels to further refine the disparity estimation in details and ill-posed regions. The proposed method is evaluated on the Scene Flow dataset, KITTI 2012, and KITTI 2015 stereo benchmarks with comprehensive experiments. Experimental results demonstrate that our method can significantly improve the accuracy of stereo matching, especially in details, occlusions, and texture-less regions.

Stereo matching is one of the main challenges in three-dimensional (3D) computer vision. Given rectified stereo-images, stereo matching is to find corresponding pixels between two images, and the disparity can be calculated for each pixel in the reference image. Accurate disparity estimation is crucial for many real-world applications such as 3D reconstruction [1], medical imaging [2], and view synthesis [3].