Coefficient of determination

Definition taught in Linear regression, part of Statistical inference

Coefficient of determination

Type Definition
Taught in Linear regression
Part Statistical inference
Worked examples 2

The coefficient of determination of a fitted line is

R^2=\frac{\widehat\beta_1^2S_{xx}}{S_{YY}}=\frac{S_{xY}^2}{S_{xx}S_{YY}},

the proportion of the variation of the y_i about their mean that the line accounts for. It lies in [0,1], and it is the square of the sample correlation coefficient of the pairs.

Examples

Height and shoe size

Worked example from Regression examples

Question. Fit the regression of height on shoe size and interpret the output.

Solution.

survey <- read.csv("studentsurvey.csv")
model1 <- lm(height ~ shoe, data = survey)
summary(model1)
# Residuals:
#      Min       1Q   Median       3Q      Max
# -18.9073  -6.1465   0.1096   6.3626  22.6384
# Coefficients:
#             Estimate Std. Error t value Pr(>|t|)
# (Intercept) 145.7776     5.7629  25.296  < 2e-16 ***
# shoe          1.0048     0.2178   4.613  1.2e-05 ***
# Residual standard error: 9.299 on 98 degrees of freedom
# Multiple R-squared:  0.1784,  Adjusted R-squared:  0.17
# F-statistic: 21.28 on 1 and 98 DF,  p-value: 1.199e-05

The fitted line is height =145.78+1.005\times shoe size: each extra centimetre of shoe goes with about one more centimetre of height, on average. Reading the rest:

  • Residuals. The five-number summary of e_i=y_i-\widehat y_i. A median near 0 and quartiles roughly symmetric about it are what a normal error model predicts.
  • Coefficients. Each estimate comes with its standard error, the t statistic estimate divided by standard error, and the two-sided p-value for the hypothesis that the coefficient is zero. For the slope, t=1.0048/0.2178=4.61 on 98 degrees of freedom, so shoe size is clearly associated with height. The test for the intercept, that the mean height is zero at shoe size zero, is meaningless here and is ignored.
  • Residual standard error. The estimate S=9.3 cm of the error standard deviation \sigma: a typical student is about 9 cm from the line.
  • R^2. Shoe size accounts for 17.8% of the variation in height. The relationship is real but explains little; most of the variation is elsewhere.
  • F statistic. The test that every slope is zero. With one explanatory variable it is the square of the slope’s t, 4.613^2=21.28, with the same p-value.

How much do the ads explain?

Question page: How much do the ads explain?

For the advertising data, S_{xx}=0.444, S_{xy}=23.34 and S_{yy}=1600.9. Compute R^2, the residual standard error S on 8 degrees of freedom, and a 95% confidence interval for the slope, using t_{8;0.975}=2.306.

R^2=\frac{S_{xy}^2}{S_{xx}S_{yy}}=\frac{23.34^2}{0.444\times1600.9}=0.766.

The line accounts for about 77% of the variation in sales. The residual sum of squares is S_{yy}(1-R^2)=374.0, so S^2=374.0/8=46.75 and S=6.84. The standard error of the slope is S/\sqrt{S_{xx}}=6.84/0.666=10.26, and the 95% interval for the slope is

52.6\pm2.306\times10.26=(28.9,\ 76.2).

Used in

See also

Least-squares line · Least-squares estimators are unbiased · Simple linear regression · Sampling distribution of the regression estimators · Slope t pivot · Prediction interval

Glossary and index