Structure-Texture Image Decomposition Using Deep Variational Priors

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Structure-Texture Image Decomposition Using Deep Variational Priors

By: 
Youngjung Kim; Bumsub Ham; Minh N. Do; Kwanghoon Sohn

Most variational formulations for structure-texture image decomposition force the structure images to have small norm in some functional spaces and to share a common notion of edges, i.e., large-gradients or large-intensity differences. However, such a definition makes it difficult to distinguish structure edges from oscillations that have fine spatial scale but high contrast. In this paper, we introduce a new model by learning deep variational priors for structure images without explicit training data. An alternating direction method of a multiplier algorithm and its modular structure are adopted to plug deep variational priors into an iterative smoothing process. The central observations are that convolution neural networks (CNNs) can replace the total variation prior, and are indeed powerful to capture the natures of structure and texture. We show that our learned priors using CNNs successfully differentiate high-amplitude details from structure edges, and avoid halo artifacts. Different from previous data-driven smoothing schemes, our formulation provides another degree of freedom to produce continuous smoothing effects. Experimental results demonstrate the effectiveness of our approach on various computational photography and image processing applications, including texture removal, detail manipulation, HDR tone-mapping, and non-photorealistic abstraction.

SPS on Twitter

  • THIS FRIDAY: Join our Vice President-Membership, K.V.S. Hari, and Membership Development Committee Chair, Arash Moh… https://t.co/rGSzhHAwgM
  • The SPACE webinar series continues tomorrow, Tuesday, 11 August at 11 AM ET with Dr. Xiao Xiang Zhu presenting "Dat… https://t.co/X5oz4KiJwX
  • now accepting submissions for special sessions, tutorials, and papers! The conference is set for June 2… https://t.co/sB3o5ItL0j
  • DEADLINE EXTENDED: The IEEE Journal of Selected Topics in Signal Processing is now accepting papers for a Special I… https://t.co/2SJwqj7aDB
  • NEW WEBINAR: Join us on Friday, 14 August at 11:00 AM ET for the 2021 SPS Membership Preview! Society leadership wi… https://t.co/1PLaZIt2VQ

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar