Classification problem
Section of Statistical classifiers and discriminant analysis, part of Statistical learning
Classification problem
| Type | Section |
| Taught in | Statistical classifiers and discriminant analysis |
| Part | Statistical learning |
The object is a rule that reads a new measurement vector and returns a class. A density estimator returns a density. A clustering method invents groups rather than using labels already in hand. The rule has to be a function of the new vector and of the labelled training set.
Definition 1 (Training data and classifier) The training data consist of multivariate observations \mathbf{x}_1,\ldots,\mathbf{x}_n\in\mathbb{R}^p of p predictor variables, stored as the rows of a data matrix X_{\mathrm{all}}, together with class labels \mathbf{y}_{\mathrm{all}}=(y_1,\ldots,y_n)^\top, y_i\in\{1,\ldots,G\}, where G is the number of classes. A classifier is a function
\eta^{(X_{\mathrm{all}},\mathbf{y}_{\mathrm{all}})}(\mathbf{x})\in\{1,\ldots,G\}
of a new observation \mathbf{X}=\mathbf{x} and the training set.
Assume initially that the labelled pairs are independent draws from a target population. Its class probabilities are \pi_1,\ldots,\pi_G, summing to one. These are called prior class probabilities because they describe an item before its measurements are observed. A prior distribution on an unknown parameter is a different object. Representative training proportions estimate them by
\widehat\pi_g=\frac1n\sum_{i=1}^{n}\mathbf{1}\{y_i=g\}.
Those proportions estimate the population priors when each case was drawn from the target distribution without fixing the class sizes in advance. A case–control sample that enrols equal numbers of cases and controls has class counts chosen by the design. Its proportions describe the sample. The priors then have to come from outside the training counts. External prevalence estimates are useful when they refer to the population in which the rule will be used.
Used in
See also
Training data and classifier · Posterior class probability · Zero–one loss and risk · Bayes-optimal rule · Fisher’s direction · Bayes classifier · Error rates and optimal classifiers · Classification with unequal costs