Accurate and Robust Video Saliency Detection via Self-Paced Diffusion

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Accurate and Robust Video Saliency Detection via Self-Paced Diffusion

Yunxiao Li; Shuai Li; Chenglizhao Chen; Aimin Hao; Hong Qin

Conventional video saliency detection methods frequently follow the common bottom-up thread to estimate video saliency within the short-term fashion. As a result, such methods can not avoid the obstinate accumulation of errors when the collected low-level clues are constantly ill-detected. Also, being noticed that a portion of video frames, which are not nearby the current video frame over the time axis, may potentially benefit the saliency detection in the current video frame. Thus, we propose to solve the aforementioned problem using our newly-designed key frame strategy (KFS), whose core rationale is to utilize both the spatial-temporal coherency of the salient foregrounds and the objectness prior (i.e., how likely it is for an object proposal to contain an object of any class) to reveal the valuable long-term information. We could utilize all this newly-revealed long-term information to guide our subsequent “self-paced” saliency diffusion, which enables each key frame itself to determine its diffusion range and diffusion strength to correct those ill-detected video frames. At the algorithmic level, we first divide a video sequence into short-term frame batches, and the object proposals are obtained in a frame-wise manner. Then, for each object proposal, we utilize a pre-trained deep saliency model to obtain high-dimensional features in order to represent the spatial contrast. Since the contrast computation within multiple neighbored video frames (i.e., the non-local manner) is relatively insensitive to the appearance variation, those object proposals with high-quality low-level saliency estimation frequently exhibit strong similarity over the temporal scale. Next, the long-term common consistency (e.g., appearance models/movement patterns) of the salient foregrounds could be explicitly revealed via similarity analysis accordingly. We further boost the detection accuracy via long-term information guided saliency diffusion in a self-paced manner. We have conducted...

SPS on Twitter

  • The DEGAS Webinar Series continues on 1 February when Dr. Francesca Parise presents "Tractable Network Intervention…
  • The Brain Space Initiative Talk Series continues this Friday, 27 January when Dr. Fan Lam presents "Quantitative, M…
  • CALL FOR PAPERS: The IEEE Transactions on Multimedia is accepting submissions for a Special Issue on When Multimedi…
  • As part of our 75th anniversary celebration, we're holding monthly trivia contests all year long! Enter now for the…
  • New SPS Webinar: On 13 February, join Dr. Harshit Gupta when he presents "CryoGAN: A New Reconstruction Paradigm fo…

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar