Saliency Prediction Network for 360∘ Videos

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Saliency Prediction Network for 360∘ Videos

By: 
Youqiang Zhang; Feng Dai ; Yike Ma; Hongliang Li; Qiang Zhao; Yongdong Zhang

Panoramic videos are becoming more and more easily obtained for common users. Although these videos have 360 field of view, they are usually displayed with perspective views, which needs the saliency informations for viewing angle selection. In this paper, we propose a saliency prediction network for 360 videos. Our network takes video frames and optical flows in cube map format as input, thus it does not suffer from image distorations of panoramic frames. The network is composed of feature encoding module and saliency prediction module. The feature encoding module extracts spatial and temporal features. Then these features are processed by a decoder and bidirectional convolutional LSTM for saliency prediction. To more thoroughly mine the motion information, the temporal stream of feature encoding module accepts optical flows before and after current frame. We also incorporate the global feature of video frames, residual attention and Gaussian priors into the network by considering the viewing behavior of 360 videos, which is useful for performance improvement. To evaluate the performance of our method, we compare it with three state-of-the-art saliency prediction algorithms on two publicly available datasets. The experimental result has shown the effectiveness of our method, which gets the best performance.

SPS on Twitter

  • CALL FOR PAPERS: The IEEE Journal of Selected Topics in Signal Processing is now accepting papers for a Special Iss… https://t.co/N0Dp4VBSAc
  • Join us tomorrow, Wednesday, 17 August at 10 AM Eastern as the SPS Webinar Series continues with Dr. Quiqiang Kong… https://t.co/aE2bQQAP99
  • The 2023 IEEE membership year begins today, which means that new members can join now and receive service through 3… https://t.co/WIGwc9iJCq
  • On 15 September 2022, we are excited to partner with and to bring you a webinar and roundtable,… https://t.co/we14OUl2QV
  • The SPS Webinar Series continues on Monday, 22 August when Dr. Yu-Huan Wu and Dr. Shanghua Gao present “Towards Des… https://t.co/ZkHjQLLn7L

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar