Deep Unfolding Network for Spatiospectral Image Super-Resolution

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Deep Unfolding Network for Spatiospectral Image Super-Resolution

Qing Ma; Junjun Jiang; Xianming Liu; Jiayi Ma

In this paper, we explore the spatiospectral image super-resolution (SSSR) task, i.e. , joint spatial and spectral super-resolution, which aims to generate a high spatial resolution hyperspectral image (HR-HSI) from a low spatial resolution multispectral image (LR-MSI). To tackle such a severely ill-posed problem, one straightforward but inefficient way is to sequentially perform a single image super-resolution (SISR) network followed by a spectral super-resolution (SSR) network in a two-stage manner or reverse order. In this paper, we propose a model-based deep learning network for SSSR task, named unfolding spatiospectral super-resolution network (US3RN), which not only uses closed-form solutions to solve SISR subproblem and SSR subproblem, but also has extremely small parameters (only 295 K). In specific, we reformulate the image degradation and incorporate the spatiospectral super-resolution (SSSR) model, which takes the observation models of SISR and SSR into consideration. Then we solve the model-based energy function via the alternative direction multiplier method (ADMM) technique. Finally, we unfold the iterative process of the ADMM algorithm into a multistage network. Therefore, US3RN combines the merits of interpretability and generality of model-based methods with the advantages of learning-based methods. The experimental results show that, compared with the two-step method, US3RN achieves better results both quantitatively and qualitatively, while sharply reducing the number of parameters and FLOPs. Source code will be available at

The goal of spatiospectral super-resolution (SSSR) is to transform a low spatial resolution multispectral image to a high spatial resolution hyperspectral image. High spatial resolution images are more visually appealing and contain fine image details, and hyperspectral images (HSIs) have high spectral resolution and are applicable to detect the physical structure and chemical composition of objects. Hyperspectral imaging has become an emerging scientific tool in a variety of fields, such as target recognition and tracking [1][2][3], medical image processing [4][5], and remote sensing [6][7][8][9].

SPS on Twitter

  • New SPS Webinar: On 9 March, join Mr. Sayantan Dutta when he presents "Novel Prospects of Image Restoration Inspire…
  • New SPS Webinar: On Wednesday, 8 February, join Dr. Roula Nassif for "Decentralized learning over multitask graphs"…
  • CALL FOR PAPERS: IEEE Signal Processing Magazine welcomes submissions for a Special Issue on Hypercomplex Signal an…
  • New SPS Webinar: On 15 February, join Mr. Wei Liu, Dr. Li Chen and Dr. Wenyi Zhang presenting "Decentralized Federa…
  • New SPS Webinar: On Monday, 13 February, join Dr. Joe (Zhou) Ren when he presents "Human Centric Visual Analysis -…

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar