The fitted LDA model

Topic: Statistical classifiers and discriminant analysis · more questions on this topic
‹ Fisher’s direction LDA and principal components ›

Iris setosa and versicolor are to be classified using sepal length and sepal width.

  1. State the two-class LDA model, its unknown parameters, and how they are ordinarily estimated from labelled training data.

  2. Explain why the equal-cost LDA decision boundary is linear.

  3. Explain how unequal class frequencies or unequal error costs can affect the rule.

  1. Write \mathbb{P}(Y=k)=\pi_k and X\mid Y=k\sim\operatorname{N}_2(\boldsymbol\mu_k,\Sigma) for k=1,2, with one shared positive definite covariance. Estimate the class means by the within-class averages, the covariance by the pooled within-class variation, and the priors by the training proportions when those proportions represent the deployment prevalence. For the usual unbiased pool,

S_p =\frac{1}{n-2}\sum_{k=1}^{2}\sum_{i:y_i=k} (\mathbf{x}_i-\bar{\mathbf{x}}_k)(\mathbf{x}_i-\bar{\mathbf{x}}_k)^\top.

A likelihood convention uses divisor n.

  1. The common term \mathbf{x}^\top\Sigma^{-1}\mathbf{x} cancels from the log-density difference. The remaining discriminant is affine in \mathbf{x}:

\delta_k(\mathbf{x}) =\mathbf{x}^\top\Sigma^{-1}\boldsymbol\mu_k -\tfrac12\boldsymbol\mu_k^\top\Sigma^{-1}\boldsymbol\mu_k +\log\pi_k.

  1. Priors alter the intercept of the log posterior odds. Unequal misclassification losses change the posterior-odds threshold: choose the class with the smaller expected loss. Sampling proportions should be replaced by known target-population priors when those priors are known and the sample was not drawn with the target class frequencies.