A quadratic boundary

Topic: Statistical classifiers and discriminant analysis · more questions on this topic
‹ LDA with unequal priors Parameter counts ›

Class 1 is \operatorname{N}(0,1) and class 2 is \operatorname{N}(0,4) with equal priors. Find the QDA decision boundary and explain why LDA cannot separate these classes at all.

Worked answer (click to show)

Equal means do not make the two distributions equal. The variance difference makes the centre favour one class and both tails favour the other. Equal priors and standard deviations 1 and 2 give

e^{-x^2/2}=\tfrac12 e^{-x^2/8},

so -x^2/2+x^2/8=-\ln 2, that is 3x^2/8=\ln 2 and

x^2=\tfrac83\ln 2=1.848, \qquad |x|=1.360.

QDA predicts class 1 for |x|<1.36 and class 2 otherwise: two boundary points. LDA assumes a common variance, and the class means are equal, so \boldsymbol\beta=S^{-1}(\boldsymbol\mu_1-\boldsymbol\mu_2)=\mathbf{0}. The linear discriminant function is constant and cannot separate the classes. All of the information is in the variance.