Cauchy–Schwarz inequality
Topic of Joint distributions, part of Probability · Also in Estimation, Statistical classifiers and discriminant analysis
Cauchy–Schwarz inequality
| Type | Topic |
| Taught in | Joint distributions · Estimation · Statistical classifiers and discriminant analysis |
| Part | Probability · Statistical inference · Statistical learning |
For random variables with finite second moments, (\mathbb{E}XY)^2\le\mathbb{E}X^2\,\mathbb{E}Y^2, with equality only when one is a constant multiple of the other. Applied to centred variables it gives |\rho(X,Y)|\le1.
In Joint distributions
From Joint distributions
The correlation \rho(X,Y)=\operatorname{Cov}(X,Y)/\sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)} lies in [-1,1] whenever the variances are positive and finite. Equality to \pm 1 means that one variable is an affine function of the other. The bound is the Cauchy–Schwarz inequality applied to the centred variables, or equivalently the fact that \operatorname{Var}(X+tY)\ge 0 for every real t.
In Estimation
From Estimation
It lies in [-1,1], by the Cauchy–Schwarz inequality.
In Statistical classifiers and discriminant analysis
The ratio depends on the direction of \mathbf{a} and not on its length. One route constrains \mathbf{a}^\top S\mathbf{a}=1 and differentiates (\mathbf{a}^\top\mathbf{d})^2 with a Lagrange multiplier for that constraint, where \mathbf{d}=\widehat{\boldsymbol\mu}_1-\widehat{\boldsymbol\mu}_2. The stationary equation is (\mathbf{a}^\top\mathbf{d})\,\mathbf{d}=\lambda S\mathbf{a}, so \mathbf{a} is proportional to S^{-1}\mathbf{d}. The same direction falls out of Cauchy–Schwarz, and that argument makes the positive-definiteness assumption easy to see.
Used in
See also
Markov’s inequality · Chebyshev’s inequality · Jensen’s inequality · Random variable · Sample correlation coefficient · Joint probability mass function · Independent random variables · Joint probability density