Deep Internal Learning: Deep learning from a single input

Deep learning, in general, focuses on training a neural network from large labeled datasets. Yet, in many cases, there is value in training a network just from the input at hand. This is particularly relevant in many signal and image processing problems where training data are scarce and diversity is large on the one hand, and on the other, there is a lot of structure in the data that can be exploited. Using this information is the key to deep internal learning strategies, which may involve training a network from scratch using a single input or adapting an already trained network to a provided input example at inference time. This survey article aims at covering deep internal learning techniques that have been proposed in the past few years for these two important directions. While our main focus is on image processing problems, most of the approaches that we survey are derived for general signals (vectors with recurring patterns that can be distinguished from noise) and are therefore applicable to other modalities.

Introduction

Deep learning methods have led to remarkable advances with excellent performance in various fields, including natural language processing, optics, image processing, autonomous driving, text-to-speech, text-to-image, face recognition, anomaly detection, and many more applications. Common to all the above advances is the use of a deep neural network (DNN) that is trained using a large annotated dataset that is created for the problem at hand. The used dataset is required to represent faithfully the data distribution in the target task and allow the DNN to generalize well to new unseen examples. Yet, achieving such data can be burdensome and costly, and having strategies that do not need training data or can easily adapt to their input test data is of great value. This is particularly true in applications where generalization is a major concern, such as clinical applications and autonomous driving.