Physics-Driven Machine Learning for Computational Imaging

Recent years have witnessed a rapidly growing interest in next-generation imaging systems and their combination with machine learning. While model-based imaging schemes that incorporate physics-based forward models, noise models, and image priors laid the foundation in the emerging field of computational sensing and imaging, recent advances in machine learning, from large-scale optimization to building deep neural networks, are increasingly being applied in modern computational imaging. A wide range of machine learning techniques can be applied to enhance the effectiveness and efficiency of computational imaging systems, thus redefining state-of-the-art computational imaging algorithms.

Recent years have witnessed a rapidly growing interest in next-generation imaging systems and their combination with machine learning. While model-based imaging schemes that incorporate physics-based forward models, noise models, and image priors laid the foundation in the emerging field of computational sensing and imaging, recent advances in machine learning, from large-scale optimization to building deep neural networks, are increasingly being applied in modern computational imaging. A wide range of machine learning techniques can be applied to enhance the effectiveness and efficiency of computational imaging systems, thus redefining state-of-the-art computational imaging algorithms.

Physics-driven machine learning has become an integral part of computational imaging

In contrast to traditional imaging methods, in which images are directly captured by the sensing device, computational imaging involves an imaging system in which computation plays a vital role in the image formation process. In particular, computational imaging can exploit the underlying physics of the imaging modality and domain knowledge, which needs to be exploited and combined with various data-driven approaches to benefit the imaging process from sensing or data acquisition to image reconstruction. There are compelling challenges for such interdisciplinary research that remain to be addressed, ranging from modeling and machine learning algorithm development and developing provable guarantees to novel imaging applications.

The main focus of this special issue of IEEE Signal Processing Magazine is on recent developments in physics-driven machine learning techniques that can be applied for computational imaging. Ten articles selected from the original 47 submissions are accepted in this issue, covering key theoretical topics ranging from model-based methods, such as sparse and low-rank representations and phase retrieval, to more advanced physics-informed deep learning, such as plug and play, generative models, and unrolling-based image reconstruction. Thanks to the tremendous interest from the research community, there are also many contributions for interesting applications, which will be published in the second volume of the special issue in March.