The Optimal Condition Number for Leaky ReLU Function

The condition number of a network layer, the ratio of its optimal upper and lower Lipschitz constants, measures how much the layer distorts distances between inputs. Xia and Zhou proved that for standard ReLU this number is at least $\sqrt{2}$ for every weight matrix and bias vector, and that Gaussian initialization approaches $\sqrt{2}$ as the layer becomes wide. We establish the corresponding results for Leaky ReLU with leakage coefficient $\alpha \in [0,1]$. An identity, exact for each neuron and each realization of the weights, writes the Leaky ReLU squared distance as a fixed combination of two standard ReLU squared distances and one linear squared distance, so the existing concentration estimates transfer without change. The optimal condition number equals $\sqrt{2(1+\alpha ^{2})}/(1+\alpha)$: no weights or bias improve on it, an explicit layer of width $2n$ attains it exactly, and Gaussian weights approach it as the ratio of width to input dimension grows. It decreases strictly from $\sqrt{2}$ at $\alpha =0$ to 1 at $\alpha =1$, quantifying the geometric benefit of leakage.