Special Issue on Stochastic Simulation and Optimization in Signal Processing

Many modern signal processing (SP) methods rely very strongly on probability and statistics tools to solve problems; for example, they use stochastic models to represent the data observation process and the prior knowledge available and they obtain solutions by performing statistical inference (e.g., using maximum likelihood or Bayesian strategies). Statistical SP methods are, in particular, routinely applied to many and varied tasks and signal modalities, ranging from resolution enhancement of medical images to hyperspectral image unmixing; from user rating prediction to change detection in social networks; and from source separation in music analysis to automatic speech recognition.

However, expectations and demands are constantly rising and such methods are now expected to deal with ever more challenging SP problems that require ever more complex models, and more importantly ever more sophisticated novel methodologies to tackle them. This has driven the development of computation-intensive SP methods based on stochastic simulation and optimisation. This field, at the interface of SP and computational statistics, has been receiving considerable attention by researchers of late because of its capacity to handle complex models and underpin sophisticated (often Bayesian) statistical inference techniques delivering accurate and insightful results. Promising areas of research in the field include the development of adaptive block-co-ordinate stochastic optimization algorithms and of efficient simulation techniques for high-dimensional inverse problems.

This special issue of IEEE Journal of Selected Topics in Signal Processing seeks to report cutting edge research on stochastic simulation and optimisation methodologies, and their application to challenging SP problems that are not well addressed by existing methodologies.

In the survey by Pereyra et al., an introduction to stochastic simulation and optimization methods in signal and image processing is presented.

On optimization, Koneˇcný et al. present a scheme to improve to the theoretical complexity and practical performance of semistochastic gradient descent (S2GD). In Donmez et al. the problem of online optimization under adversarial perturbations is considered. Kail et al. consider the task of online data reduction and outlier rejection when large amounts of data are to be processed for inference. Verliet et al. present a randomized block sampling canonical polyadic decomposition method that combines increasingly popular ideas from randomization and stochastic optimization to tackle computational problems. Carlson et al. propose a new, largely tuning-free algorithm to address the problem of training deep probabilistic graphical models.

In the area of MCMC methods, Septier et al. consider a sequential Markov chain Monte Carlo (SMCMC) technique. Murphy and Godsill examine the use of blocking strategies for Particle Gibbs sampling schemes for high dimensional latent state space models with interacting components. Feron et al. propose an optimizationguided Gibbs sampler for models involving high dimensional conditional Gaussian distributions. Lindsten et al. present a forward backward-type Rao-Blackwellized particle smoother (RBPS) that is able to exploit the tractable substructure present in these models. Akin to the well known Rao-Blackwellized particle filter, the proposed RBPS marginalizes out a conditionally tractable subset of state variables, effectively making use of SMC only for the “intractable part” of the model. Schreck et al. introduce a new MCMC method for Bayesian variable selection in high dimensional settings.

There are several papers considering a range of applications in this special issue. Rached et al. consider the evaluation of the outage capacity (OC) at the output of equal gain combining (EGC) and maximum ratio combining (MRC) receivers. Tan et al. consider a compressive hyperspectral imaging reconstruction problem where three dimensional spatio-spectral information about a scene is sensed by a coded aperture snapshot spectral imager (CASSI). Chen et al. consider the problem of how to develop a system-wide energy and workload management policy for future sustainable data centers. Mesejo et al. consider the problem of how to measure the blood oxygen level-dependent (BOLD) signal in functional MRI (fMRI).

The editors hope that the SP community will find these papers stimulating, interesting, and useful in advancing our understanding and use of stochastic simulation and optimisation methods in SP.
Reference
Steve McLaughlin, Marcelo pereyra, Alfred O. Hero, et al.Introduction to the Issue on Stochastic Simulation and Optimization in Signal Processing. IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 10, NO. 2, FEBRUARY 2016: 221-223