Color Image Restoration Exploiting Inter-Channel Correlation With a 3-Stage CNN

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Color Image Restoration Exploiting Inter-Channel Correlation With a 3-Stage CNN

By: 
Kai Cui; Atanas Boev; Elena Alshina; Eckehard Steinbach

Image restoration is a critical component of image processing pipelines and for low-level computer vision tasks. Conventional image restoration approaches are mostly based on hand-crafted image priors. The inter-channel correlation of color images is not fully exploited. Motivated by the special characteristics of the inter-channel correlation (higher correlation for red/green and green/blue channels than for red/blue) in color images and general characteristics (green channel always shows the best image quality among the three color components) of distorted color images, in this paper, a three-stage convolutional neural network (CNN) structure is proposed for color image restoration tasks. Since the green channel is found to have the best quality among all three channels, in the first stage, the network is designed to reconstruct the green component. Then, with the guidance of the reconstructed green channel from the first stage, the red and blue channels are reconstructed in the second stage with two parallel networks. Finally, the intermediate reconstructions from the previous stages are concatenated and further refined jointly. We demonstrate the capabilities of the proposed three-stage structure with three typical color image restoration tasks: color image demosaicking, color compression artifacts reduction, and real-world color image denoising. In addition, we integrate pixel-shuffle convolution into our scheme to improve the efficiency, and also introduce a quality-blind training strategy to simplify the training process for the compression artifacts reduction task. Extensive experimental results and analyses show that the proposed structure successfully exploits the spatial and inter-channel correlation of color images and outperforms the state-of-the-art image reconstruction approaches.

Image restoration (IR) is a fundamental task in image processing, which aims at reconstructing high-quality images from distorted observations. Color images suffer from different kinds of distortions introduced during image acquisition or along the processing pipeline, including missing samples, sensor noise, compression, and transmission distortion. In order to solve these issues, many image restoration approaches have been proposed in the past years. Conventional image restoration approaches are mostly based on image priors [1]. The IR problems are formulated as iterative optimization problems and regularized by image priors, including low-rank, sparsity, patch-wise smoothness, and non-local similarity [2][3][4]. These image priors have clear physical meaning, and they are general features that can handle different IR problems. However, most of them are hand-crafted and not optimal for a specific task. Also, solving an iterative optimization problem is usually time-consuming.

SPS on Twitter

  • DEADLINE EXTENDED: The 2023 IEEE International Workshop on Machine Learning for Signal Processing is now accepting… https://t.co/NLH2u19a3y
  • ONE MONTH OUT! We are celebrating the inaugural SPS Day on 2 June, honoring the date the Society was established in… https://t.co/V6Z3wKGK1O
  • The new SPS Scholarship Program welcomes applications from students interested in pursuing signal processing educat… https://t.co/0aYPMDSWDj
  • CALL FOR PAPERS: The IEEE Journal of Selected Topics in Signal Processing is now seeking submissions for a Special… https://t.co/NPCGrSjQbh
  • Test your knowledge of signal processing history with our April trivia! Our 75th anniversary celebration continues:… https://t.co/4xal7voFER

IEEE SPS Educational Resources

IEEE SPS Resource Center

IEEE SPS YouTube Channel