Nonconvex Optimization for Signal Processing and Machine Learning

The articles in this special section focus on nonconvex optimization for signal processing and machine learning. Optimization is now widely recognized as an indispensable tool in signal processing (SP) and machine learning (ML). Indeed, many of the advances in these fields rely crucially on the formulation of suitable optimization models and deployment of efficient numerical optimization algorithms. In the early 2000s, there was a heavy focus on the use of convex optimization techniques to tackle SP and ML applications. This is largely due to the fact that convex optimization problems often possess favorable theoretical and computational properties and that many problems of practical interest have been shown to admit convex formulations or good convex approximations.

Optimization is now widely recognized as an indispensable tool in signal processing (SP) and machine learning (ML). Indeed, many of the advances in these fields rely crucially on the formulation of suitable optimization models and deployment of efficient numerical optimization algorithms. In the early 2000s, there was a heavy focus on the use of convex optimization techniques to tackle SP and ML applications. This is largely due to the fact that convex optimization problems often possess favorable theoretical and computational properties and that many problems of practical interest have been shown to admit convex formulations or good convex approximations.

In 2010--exactly a decade ago--IEEE Signal Processing Magazine (SPM) published a special issue on convex optimization and signal processing in which many successful applications of convex optimization techniques in SP were showcased. At that time, it was a common belief that nonconvex optimization problems were intractable and lacked strong theoretical properties. Nevertheless, not long after the publication of the aforementioned special issue, works have started to emerge showing that various SP and nondeep ML applications give rise to well-structured nonconvex formulations, which often exhibit properties akin to those of convex optimization problems and can be solved to optimality more efficiently than their convex reformulations or approximations. This line of study has since blossomed into a highly active research area and led to new insights into the structures and tractability of nonconvex optimization problems.