Kernel-Based Image Filtering: Fast Algorithms and Applications

PhD Thesis by: Sanjay Ghosh

PhD Advisor: Prof. Kunal Narayan Chaudhury

Associate Professir, Department of Electrical Engineering

Indian Institute of Science (PhD Granting Institute)

ABSTRACT:

Image filtering is a fundamental task in computer vision and image processing. Various linear and nonlinear filters are routinely used for enhancement, superresolution, sharpening, restoration, etc. The focus of this thesis is on kernel-based filtering that has received significant attention in recent years. The basic idea of kernel filtering is quite straightforward, namely, each pixel in the image is replaced by a weighted average of its neighboring pixels. The weighting is performed using an affinity kernel, which is generally non-negative, symmetric positive definite. Depending on the choice of the kernel function, there could be different filters: Gaussian, bilateral filter, nonlocal means, guided filtering, etc. While the dominant applications of kernel filtering are enhancement and denoising, it can also be used as a powerful regularizer for image reconstruction. In general, the brute-force implementation of kernel filtering is prohibitively expensive. Unlike convolution filters, they cannot be implemented efficiently using recursion or the fast Fourier transform. In fact, their brute-force implementation is often too slow for real-time applications. The key motivation of this work was to develop fast approximation algorithms for kernel filtering and explore their applications.

We have focused on two popularly used kernel filters, bilateral filter and nonlocal means, in this thesis. In the context of bilateral filtering, we demonstrated that by using Fourier approximation of the underlying kernel, we can obtain state-of-the-art fast algorithm for filtering of gray images. The main idea is to express the filtering as a series of fast convolutions, which are applied to simple nonlinear transforms of the input data. We achieved around 50x speedup using our proposed method. In relation to existing works, a unique aspect of our method is that we are able to analyze and provide theoretical guarantees on the filtering error incurred by the approximation. We extended this to color images, texture smoothing, low-light image enhancement, etc. In a different direction, we have developed a fast algorithm for symmetrized nonlocal means, which can be used as a regularizer (denoiser) in plug-and-play image restoration. Plug-and-play is a recent paradigm where a powerful denoiser is used to regularize the inversion of the measurement model within an iterative framework. The attractive aspect of symmetrized nonlocal means is that the associated plug-and-play iterations are fast and provably convergent. In practice, the proposal algorithm can significantly speedup various restoration tasks such as deblurring, inpainting, superresolution, and single-photo imaging--what would typically take minutes can now be done in seconds.

Sanjay Ghosh has has successfully defended the Ph.D. thesis on October 21, 2019. A final version of the thesis has also officially been submitted according to the Ph.D. degree requirements of the institute. PhD Thesis.