Sparse approximation is a well-established theory, with a profound impact on the fields of signal and image processing. In this talk we start by presenting this model and its features, and then turn to describe two special cases of it – the convolutional sparse coding (CSC) and its multi-layered version (ML-CSC). Amazingly, as we will carefully show, ML-CSC provides a solid theoretical foundation to … deep-learning. Alongside this main message of bringing a theoretical backbone to deep-learning, another central message that will accompany us throughout the talk: Generative models for describing data sources enable a systematic way to design algorithms, while also providing a complete mechanism for a theoretical analysis of these algorithms' performance. This talk is meant for newcomers to this field - no prior knowledge on sparse approximation is assumed.

Generative convolutional networks have obtained spectacular results to synthesize complex signals such as images, speech, music, with barely any mathematical understanding. This lecture will move towards this world by beginning from well relatively understood maximum entropy modelization. We first show that non-Gaussian and non-Markovian stationary processes requires to separate scales and measure scale interactions, which can be done with a deep neural network. Applications to turbulence models in physics and cosmology will be shown. We shall review deep Generative networks such as GAN and Variational Encoders, which can synthesize realizations of non-stationary processes or highly complex processes such as speech or music. We show that they can be considerably simplified by defining the estimation as an inverse problem. This will build a bridge with maximum entropy estimation. Applications will be shown on images, speech and music generation.