Properly Handling Complex Differentiation in Optimization and Approximation Problems

Functions of complex variables arise frequently in the formulation of sig - nal processing problems. The basic calculus rules on differentiation and integration for functions of complex variables resemble, but are not identical to, the rules of their real variable counterparts. On the contrary, the standard calculus rules on differentiation, integration, series expansion, and so on are the special cases of the complex analysis with the restriction of the complex variable to the real line. The goal of this lecture note is to review the fundamentals of the functions of complex variables, highlight the differences and similarities with their real variable counterparts, and study the complex differentiation operation with the optimization and approximation applications in mind. More specifically, the take-home result of this lecture note is to understand the differentiation with respect to the conjugate variable ( / 2 2z f r r ) (z z, ), which is known as Wirtinger calculus, and its application in optimization and approximation problems.

 

Relevance. Complex analysis is a rich and interesting topic with close ties to the foundation of our profession. To illustrate its foundational nature, it is important to remember that the independent variable of a transfer function, e.g., H s( ) or H z( ), is a complex variable. Hence, the transfer function, a concept at the core of many signal processing operations, is nothing but a mapping via the function of a complex variable. In the early days of signal processing, in the late 1970s to early 1980s, an electrical engineering curriculum could not be considered “good” without a mandatory course on complex analysis where differentiable functions, Taylor/ Laurent series, contour integration, and residue calculation, were presented in some depth to an undergraduate audience. With the progress of time, the attention of undergraduate students has been directed elsewhere, and topics of complex calculus have been eliminated from most undergraduate curricula. Today, with the exception of a very few students, even top graduate students do not know why the standard partial-fraction expansion method is called the residue method.