Demystifying Lie Group Methods for Signal Processing: A Tutorial

Many problems in signal processing [e.g., filter bank design, independent component analysis (ICA), beamforming design, and neural network training] can be formulated as optimization over groups of transformations that depend continuously on real parameters (Lie groups). Such problems are usually tackled in two ways: using a constrained optimization procedure or using some parameterization to transform them into unconstrained problems.

Many problems in signal processing [e.g. filter bank design, independent component analysis (ICA), beamforming design, and neural network training] can be formulated as optimization over groups of transformations that depend continuously on real parameters (Lie groups). Such problems are usually tackled in two ways: using a constrained optimization procedure or using some parameterization to transform them into unconstrained problems.

There is a third way, though. In the numerical analysis literature, it is possible to find several algorithms especially tailored for Lie groups. The description of such algorithms, however, uses mathematical concepts that are not widely known in the signal processing community and can be obscure for the practitioner. This article presents to the signal processing community these specialized techniques using only some basic analysis and linear algebra.

The objective of this article is to present some very powerful but little-known techniques for optimization over spaces that can be constructed as homogeneous spaces starting from continuous groups of transformations (Lie groups). Despite the apparently very abstract setting, this kind of problem is found in several signal processing contexts.