Superpixel Guided Network for Three-Dimensional Stereo Matching

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Superpixel Guided Network for Three-Dimensional Stereo Matching

By: 
Xiaoqian Ye; Xinzhu Sang; Duo Chen; Peng Wang; Kuiru Wang; Binbin Yan; Boyang Liu;Huachun Wang; Shuai Qi

Superpixel provides local pixel coherence and respects object boundary, which is beneficial to stereo matching. Recently, superpixel cues are introduced into deep stereo networks. These methods develop a superpixel-based sampling scheme to downsample input color images and upsample output disparity maps. However, in this way, the image details are inevitably lost in the downsampling and the upsampling process introduces errors in the final disparity as well. Besides, this mechanism further limits the possibility of utilizing larger and multi-scale superpixels, which are important to alleviate the matching ambiguity. To address these problems, a superpixel-guided stereo matching method (LSG-Stereo) is proposed, which explicitly exploits the feature and disparity consistency within multi-scale superpixels to improve disparity estimation. To effectively incorporate superpixel cues into a stereo matching network, two novel modules are designed, including Superpixel Attention Spatial Pyramid Pooling (SA-SPP) and Superpixel-Guided Refinement (SGR). The SA-SPP module takes advantage of the content-aware superpixel pooling to construct an adaptive spatial pooling pyramid for better feature extraction. The SGR module explicitly utilizes the disparity consistency over multi-scale superpixels to further refine the disparity estimation in details and ill-posed regions. The proposed method is evaluated on the Scene Flow dataset, KITTI 2012, and KITTI 2015 stereo benchmarks with comprehensive experiments. Experimental results demonstrate that our method can significantly improve the accuracy of stereo matching, especially in details, occlusions, and texture-less regions.

Stereo matching is one of the main challenges in three-dimensional (3D) computer vision. Given rectified stereo-images, stereo matching is to find corresponding pixels between two images, and the disparity can be calculated for each pixel in the reference image. Accurate disparity estimation is crucial for many real-world applications such as 3D reconstruction [1], medical imaging [2], and view synthesis [3].

SPS on Twitter

  • Join Dr. Peilan Wang and Dr Jun Fang for "Channel State Information Acquisition for Intelligent Reflecting Surface-… https://t.co/jOhyA10xuG
  • The SPS Webinar Series continues on Monday, 10 October when Dr. Luisa Verdoliva presents "Media Forensics and DeepF… https://t.co/aInDvTSQZc
  • DEADLINE EXTENDED: The IEEE Transactions on Multimedia is accepting submissions for a Special Issue on Point Cloud… https://t.co/UqoOXUd8BH
  • Short courses return to ! Register for live and remote sessions, "A Hands-on Approach for Implementing Sto… https://t.co/qMoR6iqp4F
  • Join Dr. Sabyasachi Ghosh on Wednesday, 21 September for a new SPS Webinar, “Tapestry: A Compressed Sensing Approac… https://t.co/MNhu1kBmxG

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar